I waited until my twins were 6. Your biggest concern should be the 3-year-old climbing up there when you aren't looking. I made my 6-year-olds read the warning label on the bed and in the manual. They're 10 now and there have been no incidents.
We have the early version of this one from Ikea.
http://www.ikea.com/us/en/catalog/products/20247977/
I really like the metal frame. It's a lot less creaky and wobbly than the wooden frames.