Is this too soon?!?!?!?

I agree with Steve and don't let this guy be your only option...keep your options open =] he's not the only guy...yet...with what you said..it seems like subconsciously you want him to be the only guy. Go for it and keep your eyes and heart wide open. No ones perfect...if it doesn't work out with him it'll work out with someone else. Its a win-win situation.