The wife and ex are best friends, nothing good will come of this gathering with gf lol. It's not about OP prioritizing his ex.
OP, you are absolutely right to avoid anything more than formalities with this family. Girls will be old enough to be friends without parental involvement, soon enough. Your new gf should appreciate that you're aware of situations that will be awkward. If she eventually is the one, she'll be indoctrinated into the second wives club and it's not fun. Ask me how I know. I think it's smart that you're protecting her and she'll hopefully come to appreciate this in the future.
I agree with other posters. It's a bit of a red flag that this an issue for a 41 yo woman. Keep a close eye on this, OP.
I would suggest showing her this post, but don't