Jem is also becoming painfully of what is right and wrong. Not only does he not want to be punished by Atticus, but he also wants to be able to preserve his pride. Jem is ashamed of what he has done and doesn't want anyone else (other than Scout, who already knows) to realize what he has done. He doesn't want his father, especially, to know that he has done something to be ashamed of.
At one point, Scout tries to persuade Jem not to retrieve the pants. She states that he'll only receive a whipping, but Jem does not want to be punished in that way, since his father hasn't done it before. Scout says that Atticus has just never caught Jem doing anything wrong. Jem replies, "Maybe so, but--I wanta keep it that way, Scout. We shouldn'a done that tonight, Scout." Jem's attitude makes it clear that he is growing up.
The reason that Jem returns to the Radley yard to retrieve his pants is because he does not want to disappoint his father. Jem is maturing into a morally upright individual like Atticus and is considerate of his father's feelings. Before Jem leaves to retrieve his pants, Scout begs him not to go. Jem then explains to Scout that Atticus has never spanked him before and he wants to keep it that way. Jem also realizes that he was wrong for sneaking into the Radley yard in the first place. When Scout says to Jem that she is going with him, Jem tells her "no" because she will make too much noise. Jem does not want Scout getting Nathan Radley's attention and also does not want to put her in harm's way. Jem chooses to go alone back to the Radley yard where he finds his pants mended and folded on top of the fence.
To cover his missing pants, Jem (and the other children) had to lie. This put their relationship with Atticus in danger. The missing pants are also a piece of evidence; they prove beyond doubt that the kids went to the Radley house, and that they were trespassing, etc. On a simpler level, Jem has to go because he is afraid, and he wants to face his fears.