My MIL is utterly perplexed that we aren’t visiting their house this year. We budget enough vacation time and dollars to see each side of the family once a year. This year, there is a wedding on H’s side of the family. We have to fly to get there, rent a car, and stay in a hotel. That’s more than the money budgeted for the usual trip to see his family because we stay with his parents on the normal trip.
Between a trip to see his family, one to see mine, school days off and one actual vacation, our leave is spoken for every year. He has to say “No, that won’t work for us.” to his parents all the time. Your H needs to learn that phrase too.