OP back again. We booked a long weekend in Chincoteague at a nice hotel with an indoor/outdoor pool. I'm picturing ponies, ice cream and swimming! Thanks again!

I'd also recommend adding Charlottesville to your list. It's a great college town with some fun restaurants, check university sports schedule, and spend some time playing at the Discovery Museum with the little ones. Monticello would also be pretty to see in May.

http://www.vadm.org/about.html

Harpers Ferry is another good weekend destination if you like history and hiking...can your little ones be carried in back packs if they get tired?

Chincoteague is a fun little weekend trip...wild ponies, marsh/beach walks, ice cream treats, good seafood. Plenty to keep everyone happy and busy...maybe find a hotel with an indoor pool.

oops, forgot to add the link to the festival. Never been to this one, but have been to Front Royal and small town festivals are usually a lot of fun (especially those involving wine!)
http://www.wineandcraftfestival.com/

Head to Front Royal

The Wine and Craft Festival is on May 18th. Dinosaurland is close by.

Rent a cabin together or two small ones next to each other but outside of town. Head into town to enjoy festival events and food/drink and spend the rest of the time enjoying spring in the mountains or checking out the dinos

OP here. Wow, these are all great ideas! This is why I love DCUM! I have no idea which we'll choose, but I'm already excited.

I'd say pick a place that appeals to the adults but will allow both families time and space to maintain nap/bedtimes. When our kids were young, we loved renting cabins with friends. We'd have fun with the kids just running around in the woods all day, tire the kids out, put them to bed, and then hang outside by the fire until late at night. We all loved it. Your kids are young enough that they will be happy and entertained by just being with another kid and in new surroundings.

And don't rule out a beach trip. There will be fewer bugs, pleasant temps, prices will be cheaper in May, more availability, and even though it will be too chilly to swim, it can still be fun to be in a beach town or just to be near the water...beach walks, sunsets, kite flying, etc.

Lewes Delaware could be a fun destination.Cute, quiet little town. Take a ferry ride to Cape May could be fun for the kids.
See what's available on VRBO or consider renting some of the cabins at Cape Henlopen

http://www.destateparks.com/camping/cape-henlopen/cabins.asp

I'd also recommend considering the Eastern Shore. Great seafood, quaint towns, laid back feeling, and you can easily find playgrounds along the way to keep the kids happy (along with some good ice cream stops!).

Not sure if any of you or your little ones are train fans, but Strasburg, PA could be a fun destination with a train theme:

http://www.strasburgrailroad.com/index.php

Lunch or stay at the red caboose:
http://www.redcaboosemotel.com/

Sandy River Retreat would be fun with another family:

http://www.sandyriverretreat.com/Pages/aboutus.aspx

A farm stay could be fun....Lancaster or Shenandoah Valley, both will be pleasant in May.

http://www.greenbriersf.com/

This is what I did when my kid was that young.

DH and I are looking to do a short vacation (maybe over a long weekend) with another couple. We each have a 2.5 y.o. toddler. We're looking for something within a few hours drive from DC that would be nice for the grownups and not too boring for the tots (and not crazy expensive). We would be okay with renting a house, booking a hotel, B&B, etc. The vacation would need to take place in about a month (by mid-May at the latest), which would probably rule out beaches since the water will still be too cold.

I've been trying to brainstorm ideas but I'm coming up blank. Every since DS was born, all of our vacations have centered around visiting relatives, which we are trying to avoid this time. All of your ideas are greatly appreciated!