Different generation. I never went either but it used to be completely out of reach for most middle class people.
Well I am in my 20s, and I have only been to Disneyland one time and Disneyworld another. Disneyland when I was 2, which I don't remember at all, and Disneyworld was when I was three, which I only have vague memories of. My brother was a baby, so he obviously has no memory. We went with my grandparents and never went again because my parents don't like super commercial type vacations, my mom has extreme motion sickness so she hates amusement parks, and they get panic attacks in those sorts of crowds with small children. Obviously I was curious about Disney and I wanted to go when I was in elementary school, my mom said that we were doing other vacations and that was that. We also never went skiing, which was something I was curious about due to school as well and pestered my parents about (they said it was expensive, and not their thing and we didn't go). We also never went to any other amusement park, and I didn't get a chance to go until I was older. I survived to be a well adjusted adult who does not harbor resentment towards my parents over this.
However we did have some amazing vacations growing up. We went to a few places in Central America, Nantucket, the Grand Canyon, London and Paris, and San Francisco to name a few I can remember off the top of my head. I have wonderful memories of all of these vacations. Some were beach vacations, some had jungle hikes, others were going through museums and sight seeing, but all were fantastic times to bond as a family. I don't feel particularly deprived of anything in my childhood, and feel fortunate that I was able to travel internationally from a young age as well as see some of the wonderful parts of this country. I don't think that you "need" to do anything that you don't want to do, nor do I think it's cheating your kids' childhood to never go on a Disney vacation. There are so many great parts of this world to see, and you can never see all of them. If you don't feel like the time and money spent on Disney would be as enjoyable/worthwhile to your entire family (kids AND adults) than other vacations, then just don't go.
If it is your family's thing, great. If it isn't, great. If the kids want to go, but the adults really don't and have limited time and money to travel, well kids are kids, they'll get over it. I think that spending quality time as a family is most important.