No, that would not work in either of our families. Family trumps the formality of the restaurant, so everyone, down to the baby, is invited, and the restaurant chosen is usually a good, but family-friendly one.
Your FIL must be some old-fashioned relic that believes women aren't that important, correct?
I would NOT be happy, particularly about the fact that DH didn't stick up for me and suggest a less formal restaurant were everyone could go (or suggest a sitter, or whatever works for you). It's not too late for DH to say this!
How rude. However, if your FIL is usually obnoxious, maybe it's a blessing in disguise.