Homelessness can be an invisible abstraction steeped in percentages, but a panhandler is visible and deliberately un-ignorable. When we "encounter" one (a shameful word) we immediately condemn -- "A drunk, or an addict, or can't hold down a job", etc. Then we shudder with an "There but for the Grace of God go I" and then we retreat from the smell of urine, the filthy clothes, etc. As we get older, we imagine what conditions would prevail to turn ourselves into panhandlers. Then we drop a coin in his/her cup, "knowing" it will go to cheap wine. The whole social exchange makes the human condition too raw, and the "community" want to maintain its illusion of wellbeing. So "we" must eliminate panhandlers, not from some kind of benevolent feelings toward them but to avoid the discomfort their existence implies.
I note that you have placed this question in the “Religion” section. If this is intentional and you are asking this question in religious terms, the answer will be very different than if you are asking about it from the point of view of the social sciences.
If we are thinking about homelessness and panhandling from a religious point of view, a community should care about panhandling because Christians (the majority religion in the United States) are called to treat the poor with compassion. Jesus said that whatever people do to the “least of” his brothers and sisters, they are doing to him as well. He is telling Christians that they are expected to give food and drink and clothing to those who need it. They are expected to help those in need. Homeless panhandlers are, by most definitions, people in need. Therefore, Christians need to care about them if they are going to obey the teachings of Jesus.
If we are thinking about homelessness and panhandling from a social sciences point of view, we should care about it largely because it can affect the quality of life of everyone in a community. If we cannot walk our streets for fear of aggressive panhandlers, our quality of life diminishes. If we have to worry about homeless encampments and the sanitation problems and potential crime problems that go along with them, our quality of life diminishes. In addition, (if we feel this way) we should care about homelessness and panhandling because we should want the best for our fellow citizens.
When it comes to “how” to care, there is not much difference between the religious and the social science points of view. In both cases, we should try to do things that will reduce homelessness in the long term. We should try to create mental health and drug treatment programs for the homeless. We should try to help them get jobs. We should try to make it so they can get back on their own feet and not have to rely on the kindness of strangers.