No indeed, but that idiom always bothered me, and FWIW my stock response to this is “neither does poverty”.
I’ve always found the idea wealth is mutually exclusive with certain states, like love or happiness, a very silly notion.
I don’t aspire to wealth, I only want to have enough to feel secure, and not constantly worry I will be unable to find work to live.