We have always had fun in Williamsburg. Lots of open areas for little kids to walk/run around. Some nice playgrounds in the area. Fairly affordable.
We spent on night in Frederick last summer with an 11 month old and a 3 year old. Their minor league baseball team is very family friendly, there is a local train you can ride (Walkerville...I think?), a train museum, a nice park downtown, etc.
Might do a night in Richmond this year...