Aristotle believed in the idea that there was nothing that was inherently good and nothing that was inherently bad. He believed that everything was good if it was done to the right degree. By contrast, everything was, in his mind, bad if it was done too much or not done enough. This is what he is talking about with the idea that either extreme wealth or extreme poverty can be debilitating.
It is easy to see how abject poverty would destroy us. We would not have enough to eat. We might well suffer from poor health because of our bad diet and an inability to afford medical care. We would have a hard time in our personal relationships because the poverty would stress us and make us difficult to get along with. We would worry all the time about how to take care of ourselves and our families.
On the other hand, extreme riches could also destroy us, albeit in different ways. Physically, we would probably be alright (unless we succumbed to gluttony or excessive alcohol and drugs or something like that). But emotionally and ethically we might be destroyed. We might come to value our wealth so much that we became obsessed with making more. We might stop seeing other people as people and see them only as means to make money. We might stop really caring for our family, instead spending all our time trying to make more money.
In these ways, both excessive wealth and completely insufficient wealth are dangerous. Aristotle’s idea of the “golden mean” tells us that we need to have wealth (as well as everything else) in moderation.