I’ve been in the same exactly position with a high profile family. I called the nanny agency first and even though they suggested to ride it for a few more months, I had to think about my health first and foremost.Environment so important and while working for that family I experienced horrible headaches and felt bit up mentally. Therefore, I handed over a resignation letter giving them 2 week notice out of curtesy. They were very upset and told me not to come back.
After that, the headaches were gone.

Sorry you have to go through this, do what’s best for you.