Agent from above.
OP, I think you're getting bent out of shape about something that is clearly personal to you. You're selling a house. It's a business deal. Don't let this feedback get to you or deem it unhelpful or not. It's someone else's perspective of your house and if you're chalking so much of it up to unhelpful it's probably actually something that you should note - that people are seeing your house and comparing it to others. If you don't want to fix it, don't. But then realize that whether it needs to be "fixed" or not, this is what the buyers are using to justify your price being too high.
And as for "buyers wasting everyone's time." I have to stop laughing...because you have no idea. I had someone last week demand to see a house because he wanted to write an offer on it right then. I didn't get to eat with my kids that night. I did the whole contract and he didn't sign. He said he needed to think about it. Then he said he's going to apply for a job overseas. This is all within 36 hours. So you see how the old adage "Buyers are Liars" came about, right?
Stop letting this stuff bother you. Just ignore it and move on. Your buyer is out there. Unless something shiny dances by their face before they ink the contract.