I am 16:09. OP, what you write sounds so familiar in terms of family structure. I would really talk to him about it. PEP offers babysitting, so depending on the time offered, you can make it more "fun" -- take the kids out to eat or do something before after and then go to PEP class.
The ADD could really be driving this, not just behaviorally, but also in terms of meds. Have you encouraged him to talk to the doctor again and discuss the anxiety and irritability? It could be that if he is still experiencing these to a significant degree that he needs a different ADD med (a non-stimulant? or one that is more of an anti-depressant? you should also be aware that in some ADD can be misdiagnosed and a better fit would be bipolar depression II with hypomania instead of mania and/or with dysphoria instead of euphoria. In those who are really BP and not ADD, the ADD meds can increase irritability/rage.) You can even ask to go along to the appointment to briefly speak about what you are seeing.
One thing that's really important for ADD (and all of us) is developing routines or habits. If you always do the same thing the same way, you develop a sort of "muscle memory" that helps when your "real" memory is failing. With kids it can be hard to stick to routines, but I've found it very helpful/calming in our family to do so. I also use checklists for DH to help him remember to do kid stuff. (It can come off as bossy, so I don't present it as a tool for me to use and just model using it and DH picks up on it.)
A good ADD coach should be willing to come to the house, so that shouldn't add to the feeling of being overwhelmed too much.