It's normal wear and tear. And you didn't even notice until your DH pointed it out? Anyone who tries to return something after four years of heavy use is a cheapskate in my book.
Reminds me of when my sister used to work at JC Penney in the boys dept. A man come in, threw four pairs of dirty, stained jeans on the counter and said, "What my money back." When my sister inquired WHY, he said, "These don't fit my boys anymore."
My god.