Is picking up the kids part of the everyday routine?
If it's part of the everyday job, I think it's fine. I've worked outdoor jobs where we had to work outside rain or shine, unless it was something like a thunderstorm or hail. When I was a dog walker, yup, you still go out in the pouring rain for the 30 minute walk, and you made sure you were prepared with rain gear and towels.
If they specifically asked the nanny to pick up the kids on this one day, but not any other day, I don't think that's fair.