If you were only dressing that way for "the girls," but dressed like a frump for him when you went out, he might have a point. If you dress the same way even with him, there really is no point. He has nothing to stand on.