We left briefly for at about that age for a daycare closer to our home that was good, but not great. Financial was one of our factors, but it just wasn't worth it. I immediately regretted leaving our well run, loving daycare where our son was thriving. Luckily, we were able to switch back within a few months due to kids leaving for PK3 in the District.
It was a classic grass is greener situation, and I hear you on the lack of sick days and for me I do both pick up and drop off as husband works in the other direction.
But, from the beginning I have decided to just embrace the commute as together time. I try to use it as time together like any other time where we listen to music, play random games like "count the stoplights," and talk to each other about our day. Or we just sit and relax after a long day, which everyone needs anyway.
There will always be pros and cons, but my advice is if it ain't broke, don't fix it.