Early in the novel, when they were quite young, Scout, Jem and Dill were fascinated by the mysterious circumstances of Arthur "Boo" Radley; the fact that he never appeared, coupled with the gossip and (mis)information that occasionally circulated about town whetted their imaginations and inspired all sorts of ill-conceived but creative projects designed to "make him come out". Daring each other to touch his house was one, trying to peek into his window at night was another (which resulted in gunfire, by the way, as well as the appearance on the street of a pants-less Jem), using fishing poles to send him notes yet a third. Their childlike enthusiasm was particularly amusing the day that Dill suggested that he, Scout and Jem offer to buy Boo some ice cream in order to convince him that the three children aren't going to hurt him.
In addition to trying to lure Boo out to visit with them, the kids enact what Scout refers to as "a melancholy little drama" where she, Jem, and Dill take turns playing the Radleys as they go through the dramatic events that led to Boo's seclusion. Atticus figures out what they are doing and insists that they stop; later, Miss Maudie explains some of Boo's history to Scout, which begins to demystify him in her eyes, even as she begins to outgrow some of her fascination and fear.
They didn't HARASS Boo Radley per say, but they role played him when they were acting in games and ran by his house each day in fear. In the end, Scout actually meets Boo Radley and gets to speak with him, but to get to your question: they feared Boo and that was mostly because nobody had any PROOF that he a) even existed or b) or matched the description that everyone gossipped about.