OP,
My son was like this and the issue was severe ADHD, inattentive type.
Inattentive ADHD is not well recognized so you will not receive much help from posting on this forum or even asking your general ped, who is not an expert, either.
You can post on the special needs board, but first I suggest you read up on inattentive ADHD symptoms in young children, because in order to have a fruitful discussion, you need to make a list of symptoms and use medical or psychological terms.
So:
1. Processing speed is evidently below the normal range. 1st red flag.
2. Fine motor skills are lacking. 2nd red flag.
2. Does he have problems with spatial awareness and coordination?
3. Is he often day-dreaming and inattentive, off in his own world?
4. Can he sometimes have periods of intense hyperfocus, only on topics he loves? ADHD is a dysregulation of attention, not a lack of attention.
5. Social skills? You say aloof. Does that mean inattentive and therefore not able to communicate and socialize in an age-appropriate way?
All of these would be red flags for inattentive ADHD.
If your son does have a diagnosable disorder, you will find that his symptoms become more pronounced and impactful over time. He's still at the age where parents are tearing their hair out and wondering whether they're the crazy ones, because no one except them can see the entire problem. Can you tell I've been there?
So hang in there, OP, and please have patience with him. It's very hard to teach an inattentive child without damaging the relationship with anger and frustration, but that's what you need to do.
If he does have ADHD, there will be accommodations and services you can ask from the school with an IEP or a 504 (IEP strongly recommended because more enforceable), which is a document guaranteeing help for various special needs. But, first, do your research, and then, have him evaluated by a psychologist.