It is important to remember that we are told the story from Scout's perspective, using the first person point of view. Therefore we see everything through her child-like eyes. What is key to focus on is how the children's perspective of Boo Radley begins as a childish joke almost, as they treat him like a bogeyman and try to get him to come out, but then it develops into a real relationship as Boo Radley begins to leave them things in the tree.
It is in Chapter 1 when we are first introduced to Boo Radley and the children, thanks to Dill, come up with the game of trying to get him to come out. Note how he is described:
Inside the house lived a malevolent phantom. People said he existed, but Jem and I had never seen him. People said he went out at night when the moon was down, and peeped in windows. When people's azaleas froze in a cold snap, it was because he head breathed on them. Any stealthy small crimes committed in Maycomb were his work.
It is clear that for the children at least, Boo Radley and the story of the Radleys is the stuff of legend and horror stories to keep you up at night and fill you with a delightful terror.
However, gradually, and especially when Boo Radley begins to leave them gifts in the knot-hole of the tree, it is clear that this has changed and with the maturing of Jem and Scout they are beginning to consider Boo as another human being and they treat their relationship as something that is important to them. Consider how in Chapter 7 Jem cries when the knot-hole is filled with cement because he realises that the only way he can communicate with Boo Radley is now closed.
So, clearly we as readers can see Boo Radley in a somewhat different light, whereas the children view him at first as the bogeyman of the community. However, and key to the maturing of the children, this view changes as they begin to realise that he is another human being worthy of respect.