This place isn't far. It's about as no-frills as it gets, but the seafood is fresh and you'll find all of the Eastern Shore standards on the menu:
https://shrimpboatoc.com
If you're open to takeout, consider The Captain's Galley Crab Cake Shack, which is in the strip mall with the Food Lion that's located on 611 just as you turn off Route 50. They are hands-down the best crab cakes in the area and worth seeking out if you're nearby (it's the family that owned the OG Captain's Galley restaurant using their own recipe). They have weird hours, though, so you may want to check ahead of time.
https://www.facebook.com/CaptainsGalleyCrabCakeShack/