Point Lookout in Maryland. Camping isn't directly on the beach, but there are beaches on the Bay very close by.

Check out also Westmoreland State Park near Colonial Beach Va (on the Potomac, playground, nature center, kayaking, river beach, fossil shark's teeth, boating) and Pocomoke River State Park (good for canoeing & boat fishing and day trips to beach)
If you want something close to home there is Pohick Bay which has a waterpark in summer, golfing, small boat rentals, Gunston Hall for history and Mason Neck for walking, go out for lunch in Occoquan

Just got back from Assateague, make sure you plan ahead re: reservations if its a weekend trip in summer, they book up but sometimes you can get a last minute opening.
My first trip to Assateague was in March and it was cold and drizzly, no mosquitoes. They will come out as soon as it gets warm, so probably depending in the lingering winter. The beach will be cool and windy.
There are also beachside campgrounds in the state parks between Rehoboth and OC but they are pretty wind blown, Assateague is the nicest. Also check out Cape Henlopen

Would there be mosquitos at Assateague in early April? Thanks!

If you're looking to camp in the spring, the mosquitoes will be less of an issue (maybe a non-issue).

Inlet bridge rehoboth bay. Cape henlopean. Both of the will have less Mosquitos and horse flies than assiteague

Assateague Island has a camping area right on the beach.

I know there is somewhere within easy driving distance where we can camp on a beach? Or by a river? TIA!