Jem went to retrieve his pants after he, Scout, and Dill went to get a look at Boo Radley. When he returned with his pants, he held them up to show Scout and went to sleep without a word. It wasn't until about a week later that he revealed something about that night to Scout. As they were walking home from school, Jem said that his pants were patched up and folded on the fence. Jem was bewildered by this, thinking that whoever did this had expected him to return:
Like somebody was readin‘ my mind… like somebody could tell what I was gonna do. Can’t anybody tell what I’m gonna do lest they know me, can they, Scout?
This is one of the many instances where Boo Radley does something to protect the children. He leaves things for them in the knot-hole of the tree as a friendly gesture. Every time he does something for them, he does so secretly. In Chapter 9, during Miss Maudie's fire, Boo wraps a blanket around Scout; she doesn't notice until he's long gone.