I'm going through it right now. I was a nanny to twin infants for 6 months, but I was forced to quit due to the WAHM who was constantly nagging and micromanaging my entire shift. It didn't end well but I miss the girls so much

.
It is what it is. It was a job and on to the next. You gotta think about it like that, as sad it is. The connection with the kids is NOT enough to make you stay at a job you're unhappy with.
Is the family upset with you? Angry? Did they understand? Is there anyway you can ask to visit, take your charge out for ice cream, etc? I know some families are okay with that. You can be honest with your feelings and hope the family is okay with it. If not, then you'll just have to put your energy on your future charges/job and think of your previous charge as a precious memory.