I regularly go to both Kiawah and Sea Island. I prefer Kiawah because of the access to Charleston.
Sea Island is gorgeous, but aside from the kids camp, there really isn't a lot to do with young kids. If you do go, stay at the beach club. You'll be closer to the beach and the pools. The pools are better at Sea Island for kids.
I actually think the Sanctuary is better than the Cloisters, but they are really close. The spa in Kiawah is better as is the food.
Getting to Kiawah is a little easier than Sea Island. That said, I do love both.