You are familiar. The chaos isn't, but you're still there. They'll probably be happier where you are. Packing and strangers are medium stressors. Being in a brand-new place, without their familiar people, is a major stressor.
Try to leave a small area as stable as you can (litterbox, food, a few toys and a scratching post of some kind). When you do move the cats, move their "stability items" and such to whatever space will be most stable where you're staying. Same thing when you find permanent housing.
They'll be fine.