Jem says this in response to Scout’s frustrations with Aunt Alexandra. Scout wants to invite Walter Cunningham over, but her aunt will not allow it because she says the Cunninghams are not the type of people Scout should be hanging out with. In fact, she says that “he is trash.”
Jem tries to explain the Maycomb caste system to Scout by categorizing the four types of people found in Maycomb society. According to Jem, Aunt Alexandra is trying to make Scout "a lady" by making sure she is surrounded by people who she thinks are proper. He then lists the types of people in order of respect with "ordinary" people like themselves at the top of the list. Next come people like the Cunninghams, who are poor and live in the woods but still have some sort of civility. Following the Cunninhams are people like the Ewells who live by the dump and live like animals with no rules. Finally and at the bottom of the list are black people, who most people in Maycomb see as the least respectable type of people.
Scout and Jem both disagree with Aunt Alexandra and view all people as equals. They try to figure out why things are as they are and consider literacy a factor in the hierarchical list. Scout concludes that "there's just one kind of folks. Folks," and Jem concurs. Jem believes that these unfair social beliefs are what keep Boo Radley from coming out.