It's funny, because this is the first year we've decided to stay in VA to have Christmas here, in our house, but away from all our family. We are feeling lonely and disconnected. Our kids are 4 and 5 and we have decided to move "home" this spring. We've tried to make VA "home" but all of our friends have moved back to their families now that they have had children.
Family is important, if your relationship with them is good. The older our kids get, the more we realize how much they are missing being away from the extended family.
OP, enjoy the time with your family. Every day is a blessing with them. Young children, in general, have a hard time traveling. You can't stay home forever, so you just have to work out ways to make it easier for your baby. Wear your baby, if you are comfortable doing that as it will help to calm your child when you are in the hustle and bustle.
Relax, enjoy your family, and take up any offer for free babysitting - even if it's just so you and DH can grab a quick lunch at panera.