I think tiny houses are adorable and could see having one as a personal getaway somewhere lovely (though I personally would not want to live in one with anyone else). But you should definitely talk with an architect about Montgomery County building codes. There are so many regulations, and they are constantly adding to them. I'm not sure how "bedroom" is defined in a tiny house, but MC requires 70 sq feet for the first occupant of a bedroom, 50 sq ft for each other occupant of same room.
http://www3.montgomerycountymd.gov/311/(X(1)S(0vwryu3skmgsqt45nfva0t2y))/SolutionView.aspx?SolutionId=1-3FREKW
Then overall you need 150 sq ft for first occupant of house, 100 per every other occupant.
http://www3.montgomerycountymd.gov/311/(X(1)S(0vwryu3skmgsqt45nfva0t2y))/SolutionView.aspx?SolutionId=1-3FREKT. 800 sq ft would be fine, then, for your family of four, but still -- There are all sorts of regulations that you'd never think of. Not saying it won't be possible, but you need to know what you might be getting into.
I do think the resale is something you should keep in mind. You say you'd eventually like to move back west, but you might find yourself stuck with a property that has a very limited market. Would that limit your ability to move?
Also, do you never plan to have people over for dinner? Even if you think you could just do it outdoors, that limits to you certain seasons and to the mercy of the weather. Same goes for playdates, and your kids will most certainly want to have their friends over.
Anyway, I live in Silver Spring and wouldn't mind at all having a tiny house in the neighborhood (of course we have many tiny houses around here, just no "tiny house"!). But I would wonder quite a bit about a family of 4 living in one that was smaller than about 700 sq ft. I say this as someone who grew up in a family of 10 in a 2 bedroom 2 bathroom house and don't have any romantic notions of living in such close quarters. Also, my family of 4 used to live in a 900 sq ft house with attic space and basement, and I was constantly stressed out by it. You do need to know yourself and your partner well (are you and your partner extremely neat, for example), as well as have a good sense of what your children's personalities are like.
Good luck!