I have both--a small house on a busy street.
The noise makes me CRAZY, especially in the warm weather with the windows open. You have to really crank up the TV to hear anything, and you find the noise level in the house escalating to keep up with the outside noise. HATE IT WITH A PASSION!
However, we now have a 7 month old, and all of the toys, accoutrements, etc, as well as frequently visiting relatives, and I would give my eyeteeth for an extra bedroom, playroom closet, whatever. I think my need for space far exceeds my hatred of the noise, which is saying a lot.
Either way, you are going to have a significant compromise, and I would not underestimate the potential for thinking "what were we thinking??" later. I agree with the PPs who suggested waiting a little and seeing if you can find a better overall fit.
Good luck!