Tom Robinson's primary crime was feeling sympathy for Mayella Ewell--a white woman.
"I felt right sorry for her, she seemed to try more'n the rest of 'em--"
"You felt sorry for her, you felt sorry for her?" Mr. Gilmer seemed ready to rise to the ceiling.
The witness realized his mistake and shifted uncomfortably in the chair. But the damage was done. (Chapter 19)
In Depression era 1930s Alabama, a black man was not expected to feel or show sympathy for a white person. Negroes were not considered equals and were viewed as being beneath the white man, so no black person was expected to show empathy for someone of a higher class. Of course, most of Maycomb's white citizens had no such feelings for Negroes, who they treated as lesser individuals. Such were the racist views of the time period.
Tom's "feeling sorry" for Mayella, when examined more closely, implies a judging on his part: he says, "She seemed to try more'n the rest of 'em..." This characterization of the rest of her family (particularly her father) as lazy would have been unseemly for a black man in that place in those days. As Scout pointed out, there were several strata in Maycomb: "regular people" like her family and her neighbors; the rural working poor, like the Cunninghams; non-working idlers, like the Ewells; and the black population. As much contempt as the "regular people" had for the Ewells, they would not tolerate any indication that a black person could have that same feeling about them.