There can be a variety of reasons for this.
One reason would apply if poor children have peers who are not themselves poor. When interacting with such peers, the poor children would come to realize that they are different from their peers in terms of material possessions. This may well cause them to become self-conscious, particularly in a materialistic society such as that of the US. They would then withdraw from the company of those peers.
Another reason has to do with what is known as emotion knowledge. This is, in part, the ability to "read" other people's emotions. This ability seems to be less developed in many poor children. This lack is believed to be connected to their poverty. The proposed connection is that poor children have parents who are more stressed. This stress can lead to the display of a great deal of negative emotion towards family members. Exposure to such things seems to make it more difficult for children to comprehend others' emotions. Children with this problem are more likely to withdraw from interactions with their peers.
Living in poverty impacts a lot of different aspects of a child's life. It is not just a matter of economics. Living in poverty means a child is likely not receiving adequate medical care and is more likely to feel long-term impact from lack of healthcare. Children in poverty are much more likely to come to school with little or no prior exposure to literacy or to school functions. As a consequence, they are not "school ready" and lag behind their peers before they even begin. This can be a huge stigma, being a student in the class who is in the "low reading" group or has to go to the special help teacher etc. However, the reality is, because of school zoning, it is much more likely for a school to have many poor kids rather than just one. In this way entire schools face the challenge of helping students in poverty overcome the issues that consistently make it difficult for them to learn. One example I always give my students... I was teaching in a high poverty area and I was assigning homework. As it got colder at night, I noticed my students were not doing their homework as much. I asked them what was going on? They explained that they would go to the one house on their block that had electricity regularly and sit on the porch at night to do their homework, but it was too cold now. I had not even considered that my students might not have electricity. But the repercussions of this fact for their learning, and my teaching, were very important to think through.
Simply put, I think these poor children will feel sad that they cannot afford whatever others can, probably simple things like new textbooks or nice food to eat... nowadays, many people are materialistic, and they will also be influenced and become self conscious.