Why does Richard's family have a difficult time accepting him?
Because Richard refuses to adapt to his deprived social surroundings and constantly seeks opportunities to improve himself. He is an individualist and a materialist who refuses his family's bigotry and questions every type of authority, including that of family members. The beatings that he recevies from his family only encourage his rebellious behavior. In addition, he cannot bring himself to be subservient to whites as the Jim Crow code of the day required and with which some fellow African Americans in the book comply. The meeting narrated by a flashforward at the end of chapter 1 between an older Richard with his father who still has to obey white masters is quite telling of the differences between the two men.
As Richard says at the end of Ch. 3 his attitude to life was to be "sceptical of everything while seeking everything" and directed his loyalties "to the side of men in rebellion".
Chapter 4 focuses instead on the tensions arising between Richard and Granny and Aunt Addie because of the boy's refusal of religion.
The young Richard seems to find more emotional ties with his street gangs than with his family. When his mother asks him not to fight anymore after a bad injury in ch. 3, Richard promises to do so but knows immediately that he will never honor that promise because that would make him lose his standing in the gang. And, he concludes, "gang's life was my life".