"The Rockpile" opens with a depiction of the natural rock formation that lends the story its name. This rockpile is situated just across the street from the apartment where John Grimes and his African-American family reside. John's half-brother, Roy, occasionally plays there and observes other African-American boys engaging in fights on the rockpile. Elizabeth, the mother of John and Roy, has strictly forbidden them from approaching the rockpile, a rule that John, who is fearful of the rockpile, has no trouble adhering to. Every Saturday, John and Roy have a routine of sitting on their fire escape to watch people from their church, whom they see as redeemed, and others, whom they view as sinners, walking down the street. Their neighborhood is fraught with hazards, including the Harlem River, where a boy once drowned.

One Saturday, while John and Roy are on the fire escape, John is drawing a picture, and Roy is feeling bored. When some of Roy's friends call for him, Roy decides to go downstairs, which makes John anxious as he fears their mother will discover it. Roy persuades John not to inform their mother and then sneaks outside. John becomes engrossed in his drawing and does not notice Roy's absence for a while. When he finally looks up, he sees a gang fight on the rockpile and watches in horror as Roy, who is at the top, gets hit with a tin can. The can cuts open Roy's forehead and knocks him down. John alerts Elizabeth, who, along with her church friend Sister McCandless, brings Roy back to the apartment and tends to his wound. The women question John, who says Roy claimed he would return in five minutes. This response does not satisfy McCandless, who fears that Gabriel, John's stepfather and Roy's father, will be furious with John. She intercepts Gabriel on the stairs as he returns from work and informs him about Roy's injury.

Gabriel enters the apartment, extremely worried about Roy, his favorite child. Roy is distressed and begins to cry as he attempts to explain what happened. Elizabeth tries to speak on Roy's behalf, but Gabriel becomes angry with her and mocks her physical appearance. He also scolds her for not keeping an eye on Roy, although she protests that it is impossible to manage all her chores and watch over Roy, who is strong-willed. Gabriel then attempts to place the blame on John, but Elizabeth defends John and prevents him from being punished, asserting that Roy's injury occurred because Gabriel spoils him. Gabriel glares at Elizabeth with intense hatred, which frightens her. She regains her composure and exits the room.