I tell my kids not to play with kids who are being mean to him, so I think he's handling it perfectly. Can you create other opportunities for him to play soccer with kids more his size?
If he's having fun generally and just occasionally they tease him, I'd encourage him to go play but leave it as his choice.
Yes, kids say mean things all the time. It's normal.