It is his responsibility and he has a solution. If he usually does pick up and drop off then it is also his responsibility to get the kids there or arrange to have them picked up.

It really isn't about you. He can have who he wants babysit them during his time the same way you can. If you are going out for the evening do you call him first to ask if he wants the kids or if he approves of the sitter?

His weekend, his time, his kids, his plans, his responsibility. Unless you have a safety concern, stay out of it.

you should not be taking the kids over there. it is his time and he need to pick them up himself or arrange transportation for them to accommodate his schedule. you should not have to do anything.