Why aren't you communicating clearly what your plans are? If you get home early and want to go for a run, or change your clothes and do some errands, or take a nap, or whatever, just say so. If you're just awkwardly hanging out with nanny and kids she probably feels weird herself, like she's infringing on your family time. Just be the boss and tell her what you need from her.