Money has nothing to do with this. It's just what the society is now tuned to believe. For e.g. I earn & save x times more than a friend of mine. He earns less, saves even lesser. But he spends the money travelling the world and enjoying life. Yes, he stays in hostels instead of hotels when he is travelling, hitchhikes instead of renting cars but then money is never the issue for him.
You need not worry if you have a declining bank balance. You need to worry if the skills to increase the bank balance decline.
More than often, if you have decent money, your life is pretty good.