I would find that intrusive and probably say no thank you as politely as I could. Later in the relationship it would be different, but I would feel very uncomfortable trying to make sure your kids didn't feel neglected, my kids were on their best best BEST behavior, trying to find mutually agreeable small talk, etc. Once I was more bonded with my NK, had a more solid relationship with you and knew you better I might be open to it, depending on whether I thought you would like my spouse and kids vs. judging them and me.