Yesterday, I took my kids to the Mount Vernon Gristmill. We had the entire place to ourselves. We had a picnic outside under the trees followed by private tours of the mill and the distillery. It was remarkably pleasant and educational and none of the crowds that were swarming around the larger Mount Vernon offerings down the road.
http://www.mountvernon.org/visit-his-estate/plan-your-visit/distillery-amp-gristmill
Other things that are on our list of upcoming family "field trips":
National Pinball Museum
http://www.nationalpinballmuseum.org/
National Museum of Health and Medicine (not for young kids or those with sensitive stomachs)
http://www.medicalmuseum.mil/
WoodEnd Nature Sanctuary, Chevy Chase (check events listing for cool family classes/hikes)
http://www.audubonnaturalist.org/index.php/about-ans/sanctuaries/woodend-chevy-chase-md
Calvert Cliffs for Fossil Hunting
http://www.dnr.state.md.us/publiclands/southern/calvertcliffs.asp
Annmarie Sculpture Garden and Arts Center
http://www.annmariegarden.org/
Claude Moore Colonial Farm and their Market Fair
http://1771.org/?page_id=940
Earth Treks Climbing Gyms
http://www.earthtreksclimbing.com/
Free Summer concerts at Strathmore
Smithsonian Folklife Festival
Maryland Shakespeare Festival