It is clear that from his very first introduction into the novel, Rufus is a figure that is going to be bullied by the other kids at Kenny's school. As Kenny comments in Chapter 3, there were two things that were "wrong" with him that made him an easy target.
Firstly, his accent and the way that he came onto the bus saying "Hiya, y'all" made him famous for his different manner of expressing himself. Kenny tells us that even though Rufus tried modifying his accent, nobody would let him forget it.
Secondly, the clothes that Rufus wears are something that makes him a target. It is clear that Rufus and his brother Cody come from a poor family, as is made clear through what they wear:
It didn't take people too long before they counted how many pairs of pants and shirts Rufus and Cody had. That was easy to do because Rufus only had two shirts and two pairs of pants and Cody only had three shirts and two pairs of pants. They also had one pair of blue jeans that they switched off on; some days Rufus wore them and some days Cody rolled the legs up and put them on.
Thus their lack of clothes and the way that they exchange them between themselves means that Rufus and Cody are easy targets for the school community, who mercilessly make fun of his background and his poverty.