Boo does not say much in the book, and he never explains why he does not leave the house. His role is more symbolic than anything else. He does stay inside because he fears the outside world, and the outside word treats him with revulsion. The people of the town are fascinated with his story, because they see him as the monster under the bed. There is some evidence that he was abused and has psychological problems. Instead of going to jail for the scissor attack, he went into a sort of house arrest.
Well, I am sticking by what I said in general. But I did just find a quote that speaks to this. In the last sentence of Chapter 23, Jem and Scout are talking about Aunt Alexander's comments about the Cunninghams and how they may be good people but Scout is not to invite Walter over for dinner. Jem says, "I think I'm beginning to understand why Boo Radley's stayed shut up in the house all this time . . . it's because he wants to stay inside." Since this is in direct reference to the hypocrisy and prejudice of the people in Maycomb (and Aunt Alexandra's prejudice), it suggests that Boo was staying inside to avoid that prejudice and hypocrisy. But this is Jem's idea and not Boo's, so we really can't even know Boo's justification. Still, the possibility is definitely presented here.
I don’t think there is any quote in the book that can directly indicate that Boo stayed inside because he wanted to avoid the outside world’s hypocrisy and prejudice. Boo was the product of an abusive childhood and his imprisonment in his own home has to be seen as a sentence imposed by his relationship with his parents and only later, as a matter of habit and institutionalized fear, did Boo probably, of his own will, choose to remain hidden in that house. At the end of chapter 14, Scout asks Dill why he thinks Boo has never run away. (Dill has a habit of running away and his imagination reinforces this playful idea of escape). Dill responds that maybe Boo has nowhere to run away too. This is certainly the case. He has nowhere else to go. After so many years, that house is all he knows.
I think that Boo recognizes that he is an outcast in town and therefore, by getting back to the outside, social world of the town, he would be faced with their prejudgments of him in spite of their own hidden problems. So, while there’s no quote that notes Boo is avoiding society because of prejudice and hypocrisy, it is understood (and he understands) that if he were to rejoin society, he would (at least initially) be faced with such prejudice and gossip-induced stories about him. So, yes, it is just implied.