Jem goes to retrieve his pants the night he lost them after returning to his house, so that Atticus will not know that they were missing.
Scout and Jem are obsessed with Boo Radley. At first, they are just curious about him. After a while, when Dill joins the group, they decide that they want to make Boo come out. The children empathize with Boo, and want him to feel included in the neighborhood.
The Radley Place fascinated Dill. In spite of our warnings and explanations it drew him as the moon draws water, but drew him no nearer than the light-pole on the corner, a safe distance from the Radley gate. There he would stand, his arm around the fat pole, staring and wondering. (Ch. 4)
This is at the point where Dill decides to communicate with Boo by sending him a note. Jem goes onto the Radley porch, but in the process is scared off by Nathan Radley, who thinks he is a prowler. Unfortunately, Jem’s pants get stuck and he decides to leave them there.
The children come up with a story to tell Atticus and Aunt Rachel and explain Jem’s pants, saying they were playing strip poker with matches (because they would not be allowed to play with cards). Yet Jem is supposed to go get his pants.
Later that night, Jem explains to Scout that he has to go get them. She is shocked. Nathan Radley is out there with a gun shooting at anything that comes into his yard, or so the children think. They feel that Jem will be risking his life to go get those pants. Jem explains that Atticus considers him an upstanding and honest son, and he doesn’t want to do anything to jeopardize that impression.
I was desperate: “Look, it ain’t worth it, Jem. A lickin‘ hurts but it doesn’t last.
You’ll get your head shot off, Jem. Please…”
He blew out his breath patiently. “I—it’s like this, Scout,” he muttered. “Atticus ain’t ever whipped me since I can remember. I wanta keep it that way.” (Ch. 6)
When Jem finds his pants, they have been mended inexpertly. He shows them to Scout but does not tell her until later that he found them waiting for him. He realizes that Boo Radley left them for him, and is looking out for him—and them—and that they did, in fact, get Boo Radley to come out that night.
This incident is a turning point for Jem and Scout’s relationship, because it shows that Scout does not understand Jem’s motivation. He is older than her, and what matters most to him is Atticus’s impression of him. It is also at this point that Jem realizes that Boo Radley is their friend.