We own a house there and go every year. Plenty to do. I'd rent a big house with a pool.

We usually rent a house on the beach at Ocean City, NJ for a week (fun with kids) and then go for 2 days to Cape May. You also can do day trips there from OC.