As the above answer states, the Radley place is a source of endless fascination and intrigue for Scout, Jem and Dill because it is so mysterious. It provides something for their imaginations to feed on in the sleepy little town of Maycomb. The house itself looks forbidding, the Radleys do not invite confidences from anyone in the town, and it is all the easier for gossip and legend to spring up around the reclusive figure of Boo Radley. The children weave the scraps of rumour that they hear into their own 'melancholy little drama', when they enact the whole grim story of the Radleys (as they see it). The Radley place is also close enough to Jem and Scout's house for them to see it frequently, which sustains their interest.The fact that the children never get a chance to see Boo clearly also increases their tendency to invent fantasies about him.
The way that Scout first introduces Boo Radley to the story is interesting. She states directly that:
In the house lived a malevolent phantom. People said he existed but Jem and I had never seen him.
Boo, then, is straightaway introduced as 'a malevolent phantom'. Of course, it is the adult Scout narrating the novel, but she keeps the fears and imaginings and wonder of her childhood days very much to the fore of her narrative, as seen here.