For now, I have swore off most carbs, although occaisionally will go to town on one sweet potato for myself (while family eats white potatoes).
If the only carb I ever ate was an occasional sweet potato, I would probably weigh <100 pounds. The one time I did a low carb diet (after my second baby), I lost 10+ pounds in 4 weeks and didn't have the energy to exercise. My body absolutely did not get used to it.
I don't really eat sweets, processed carb-y snacks, and most baked goods but I certainly eat potatoes, whole grain breads, oats, rice, regular pasta, pizza, etc. and have no trouble maintaining a healthy weight in middle age.