Before the action in Chapter 5, Scout and Jem regard Miss Maudie as a "relatively benign presence," as she allows them to play in her front yard, eat her scuppernongs, and roam about her expansive back lot. They don't speak to her much, afraid of upsetting the "delicate balance" they have achieved with her.
However, in Chapter 5, Jem and Dill's exclusion of Scout pushes Scout closer to Miss Maudie, who spends a great deal of time outside tending to her garden. She also makes Scout, Jem, and Dill each a small cake whenever she bakes a large cake. Scout, Jem, and Dill come to regard Miss Maudie as their friend because, as Scout says, "she had never told on us, had never played cat-and-mouse with us, she was not at all interested in our private lives." They trust her, and Scout turns to Miss Maudie to ask her about Boo Radley. Miss Maudie explains that Boo's father is a "foot-washing Baptist" who disdains anything pleasurable and that many of the rumors about Boo that have been spread around town are false.
Chapter 5 is when readers learn what Jem and Scout have been thinking and feeling about Miss Maudie over a long time; Scout essentially summarizes their relationship. On the most basic level, they like her. On a slightly more complicated level, they treat her as a surrogate mother or grandmother in some ways. They ask her about things (like Boo Radley), and they trust her answers. She gives them things, like some of the cake she baked. And finally, they view her as a precious part of their neighborhood/community. It wouldn't be going too far to say they love her.