#1. The quality of the neighborhood is very important to how well you'll enjoy the environment, and to eventual resale.
And, building from scratch is not for the faint-hearted. Beyond trying to envision something from paper plans, which may not end up being what you imagined, you'll be dealing with all the uncertainties associated with building - builder reliability and quality, timelines which may be optimistic, the need to spec out every single detail from flooring to door hardware, to tiles, backsplashes, etc., etc, ad nauseum, which is time consuming and can be stressful. Someone else already mentioned the septic system. Avoid septic and well water if at all possible. It's bad for resale, exposes you to risks of those systems failing for a variety of reasons, and you'll have maintenance requirements you can avoid with public water and sewer connections.
A 10-minute commute for one of you would be nice, but the trade-offs to acquire it involve plenty of risk and the potential for less satisfaction in other aspects of the property in the long run.