In the beginning of Harper Lee's To Kill a Mockingbird, Scout and Jem, and later their friend Dill, are obsessed with the neighborhood recluse and local legend: Boo Radley. For most of the novel, the children operate under the assumption that Boo hasn't left his house in years because he's a prisoner under house arrest (many tall tales discuss the various crimes he's committed, for instance), and much of their energy is spent trying to catch a glimpse of him. However, toward the end of the novel, Jem changes his opinion of his neighbor: Boo isn't being kept inside as punishment for his crimes, Jem argues; rather, he voluntarily stays inside in order to avoid the oppressive nature of the world outside.
Jem's theory is a good one, especially considering its context. It comes on the heels of the Tom Robinson trial, a perfect example of the injustice, hypocrisy, and prejudice that governs much of the adult world. After this experience, Jem is better able to understand why someone might not want to participate in society in general. While it's unclear how true this statement is, whether or not Boo actually feels this way is beside the point. Indeed, what's really important here is the symbolic meaning that's communicated. Through this theory, Jem sets up Boo to be a symbol of the rejection of contemporary society. As such, Lee suggests that it might be necessary to reject the unjust society we live in if we are going to fix it at all.
He thinks Boo doesn't come out because he doesn't want to. He says this after a discussion about all the different kinds of people in the world. He realizes the problem of having different classes of society and for the first time looks at the world through a hermit or recluse's eyes. He thinks he understands that Boo keeps inside so he doesn't have to get mixed up in all of that garbage that the other people struggle with in terms of relationships across class lines.