It won't be bad. kids adjust faster than adults. We took kids 3 and 6 last summer and they were fine. We arrived around 1pm and kept them up until 8pm local. They were tried and slept until 5:30am the next day; oddly I was up before them

After that they were good to go and pretty much on their regular schedule. Coming back is easier if you have a day or 2 before they have to be up early for school because they may need to stay up a little later and sleep in some.