It's been a few years since I headed down that way (had family there and once lived there) but I found that getting on the road before 10am worked okay. So probably to be safe, I would leave at 9 am.
When I lived there and my kid was small, I used to take her over to walk around the campus at Mary Washington U. There was a fountain there that she just loved. (Lame, I know, but little kids are fascinated by such simple things.)
Also, Carl's Ice Cream, an old-fashioned soft serve place is a local institution.
http://www.hollyeats.com/Carls.htm You might also check out Sammy T's for lunch. Laid back easy going place.
http://www.sammyts.com/ And I believe that Goolrick's, an old fashioned soda fountain is still going strong:
http://www.yelp.com/biz/goolricks-modern-pharmacy-drugs-fredericksburg
Oh and you can stop by the library and look at some picture books.
http://www.librarypoint.org/
I'm sure there's a lot more there for kids to do now but it's been a long time since I've been there. It was tough living there for a number of reasons but there were many aspects of Fburg I just loved.