I used to go to Ocean City, MD regularly around the end of February, and over to Assateague while in OC. There are lots of hotels along the beach/boardwalk that would be fun to stay at with your DS. Find one with an open pool

There are lots of shops and things open on the boardwalk, even during the winter/early spring. Ripleys Believe it or Not is open (I've never been, but I was at OC last weekend and there were people going in and out). There are also the outlets near OC.
I've always seen horses on every visit to Assateague; sometimes large groups, and close, sometimes just one off in the distance. It's a lot of luck. Make sure to dress warmly since the wind can be cutting. There's a nice visitor center that's worth stopping at, and the park itself has 3 trails you can go walk; the dunes, the woods, and the marsh. All are relatively short and easy to do; good for seeing horses.
I've never been to Chincoteague other than to the town to get ice cream, so I can't speak to that.

Have a good trip!