As others have said, perhaps the issue here is that you did a neuropsych exam, which are generally not considered reliable at this age, instead of going to a developmental pediatrician?
We went to KKI at around age 3.5 for a diagnosis. My son's hyperactivity was high, but not a danger to himself. However, his impulsivity and personal space issues were a huge problem. We started at a preschool when he was 2.5, and 5 months later had to move cross country. Thank goodness we did because they were getting ready to kick us out if we didn't pay for a full time shadow. We moved to DC and ended up in a very supportive school. But my son, since forever, has always been the kid who had "special needs" whenever he is among other children. In that, for years, you could not leave my son with other children because he would scare or hurt them. The developmental pediatrician doing the diagnosis said he had severe adhd but not as severe as some kids she sees. Had he been a genuine danger to himself or others (she used the example of the kid who tries to get out of their carseat and out of the car in a moving vehicle), we would have medicated at age 3. Because his "danger to others" was not injuring other kids (it was hitting, etc, but not hard enough to inflict injury) the ped said to wait.
Shy of his 5th birthday, we had reached our threshold with school drama. I went to my ped with KKI diagnosis in hand and he described it as the "Cadillac" of testing. He said, had I come in with concerns of ADHD, he would have sent me through a lot of hoops to flesh out the issue, but with the KKI diagnosis we were already there.
My son was legitimately different than other kids at at 2, 3, 4, and 5. (I posted in another thread that he was different at 3 weeks old, but that's another story). Other kids can look more hyper and all over the place, but my kid was the only one with legitimate "special needs" so to speak. We are medicating now and it has been wondering. The same spazy, hyper kid but like a whole new person.
So yea, not hard to diagnose kids at age 3, 4, and 5 who have moderate to severe adhd. However, I cannot fathom that any parent of a kid with mild adhd would raise the issue with a doctor or psych. If the only issue is that the kid moves around a lot or has trouble focusing, does a parent really seek out a whole evaluation?? So when people ask their doc at age 4 about adhd, I assume these are by definition going to be the moderate to severe cases and should be taken seriously.
I think the mild cases start cropping up more in academic settings because -some- parents want a reason why their kid isn't doing well in school. But no similar motivation exists in preschool.
In sum, talk to your ped to see if she thinks adhd can never be diagnosed at this age. If they think the answer is that it cannot, then yes, seek a new doc. If they just don't like the neuropsych, then that's a reasonable answer.