OP - I had the same problem. It took me years to get comfortable spending some money. And when I say spending, I don't mean big purchases. I mean going to the sales rack at stores and buying a few things and not feeling guilty about it. I still only shop the sales rack or TJ Maxx type stores, even when I was making over 200K/yr. And like you, I had zero debt, and at the time, no kids.
I think living frugally can be a great thing. But people also need to enjoy their lives and not worry too much, especially for your DH's sake. Let him enjoy his lunches and little splurges. You will create tension in your relationship if you don't back off. My DH is not as frugal as I am, but he does see the need to be frugal. So, he does try to be. We discuss big purchases but I don't nickle and dime him about little things, like eating lunch out.
Sounds like you are more than well-prepared for any emergencies with $500K in the bank. You need to find balance. I know where you are coming from as I grew up LMC, and my father was a huge penny-pincher. I know why he was since we didn't have much. One of many things I learned from him was to save and never buy on credit, which I still follow to this day. But unlike my dad, I have a heck of a lot more saved now than he ever did in his life, so I'm feeling pretty comfy.
Try taking baby steps, like buying a used book or two per month, or expensive coffee once a week or month. Just don't go over-board.

Plus, when you die, you can't take it with you, so live a little.
-signed, a reformed miser, penny-pincher