Jem is frightened when he gets his pants back because they have been mended, irregularly, and he is pretty sure Boo Radley did it.
Jem loses his pants when he is trying to spy on the Radley House, "struggling in the fence, kicking his pants off to get loose" (ch 6).
He is afraid to go back because Nathan Radley supposedly will shoot whoever comes by. Jem knows he has to get the pants though, because if he doesn’t Nathan Radley will find them and tell Atticus, and he will know that Jem was snooping around.
Jem, Dill, and Scout are fascinated with the Radley place, and Boo Radley in particular. On this night, Jem snuck onto the porch and lost his pants, and then told Atticus and the neighbors that he lost them playing poker with matches, because he would get in less trouble for matches than cards.
Jem does not want Atticus to find out about the pants because he knows Atticus will be disappointed in him. When he gets the pants back, he realizes that they have been mended.
They'd been sewed up. Not like a lady sewed 'em, like
somethin' I'd try to do. All crooked. It's almost like-" (ch 7)
Jem is shocked and frightened because he thinks that someone is in his head, knowing what he will do before he does. The fact that anyone, especially Boo Radley, knew him that well frightens him.
Later, Jem has a change of heart. When Scout finds a blanket on her shoulders as Miss Maudie’s house is on fire, he begs Atticus not to tell on Boo. He appreciates that Boo has done nothing but try to help them, and was trying to keep Jem out of trouble when he mended the pants. It is a nice gesture from Boo, and we begin to see him differently as well.