I'm a MB, and when the grandparents are coming I will let you know if they are the ones who are allowed to change the baby or be unsupervised with her.
If it's the set that is allowed unsupervised access to her, then I will tell you if they say you can go home early one day, then it's okay (and you will get paid for the full day). They will probably not ask you to come with them while they take the baby out, but may ask you for directions on how to get places. I would appreciate you giving them directions.
If it's the grandparent not allowed to be alone with the baby, you will have to be firm that I am your boss and I did not say you could go, no matter if that grandparent dismisses you. You will get a tip at the end of the week for dealing with that shit.
I will back you up. You will back me up. The trusted grandparents will respect you and treat you mostly as I do.