While there is nothing wrong with an infant under 6 months old relying on Nanny to entertain him within the house and yard, 6 months + do benefit from socialization and outings.
As a nanny, I often take infants on outings and not so I can socialize. Sure, it's fun to go to the zoo with other nannies and their charges, but I have also gone with just a 9 month old, and enjoyed myself because the baby was having so much fun.
I was homebound part time with an infant once, and by about 9 months old, she had tired of the toys she had and her little playroom. She would play for about 15 minutes, and then look at me, sign "all done" and crawl out of the playroom. That is developmentally normal behavior. Babies get bored too!
So, figure out a solid time frame in which you will do what is needed to feel as comfortable as possible with your nanny driving your child. Explain your timetable and what you will want to do to evaluate nanny's driving (riding with her a few times, or even paying for her to take a defensive driving class and paying for her time spent in class) to the candidates you interview in person. (You will have made the no driving issue clear in your ads!) Then move forward from there, and allow driving when you have said you will.