We'll find that description of the church in Chapter 12, when Calpurnia is getting ready to take Scout and Jem with her to attend a service. After Scout's initial description of the building, she describes the furnishings inside:
The churchyard was brick-hard clay, as was the cemetery beside it. If someone died during a dry spell, the body was covered with chunks of ice until rain softened the earth. A few graves in the cemetery were marked with crumbling tombstones; newer ones were outlined with brightly colored glass and broken Coca-Cola bottles. Lightning rods guarding some graves denoted dead who rested uneasily; stumps of burned-out candles stood at the heads of infant graves. It was a happy cemetery.
The warm bittersweet smell of clean Negro welcomed us as we entered the churchyard — Hearts of Love hairdressing mingled with asafoetida, snuff, Hoyt’s Cologne, Brown’s Mule, peppermint, and lilac talcum. [...]
First Purchase was unceiled and unpainted within. Along its walls unlighted kerosene lamps hung on brass brackets; pine benches served as pews. Behind the rough oak pulpit a faded pink silk banner proclaimed God Is Love, the church’s only decoration except a rotogravure print of Hunt’s The Light of the World. There was no sign of piano, organ, hymn-books, church programs — the familiar ecclesiastical impedimenta we saw every Sunday. It was dim inside, with a damp coolness slowly dispelled by the gathering congregation. At each seat was a cheap cardboard fan bearing a garish Garden of Gethsemane, courtesy Tyndal’s Hardware Co. (You-Name-It-We-Sell-It).
The first thing you'll notice in Scout's description of the church is that it was literally purchased with the money that former slaves had earned. That makes the church itself a powerful symbol of Maycomb's black community's devotion to their faith.
To find evidence of the themes of courage and resistance, we'll have to look at the details of the church building itself. The building is old and in poor repair. The church is "ancient." The paint is peeling off of it. Inside, everything is rough, faded, and sparse. The yard is made of clay, not grass. Still, the interior is welcoming, inviting, full of the spirit of community and the faith that unites it. It smells good. It's lively. Even the cemetery, with its graves for dead infants, is "happy."
All these details speak to the idea that for these churchgoers to continue in their beliefs and gather as a community to support each other takes courage and strength. The people, like the church building itself, resist abrasive forces of society and nature, respectively, that are working against them. For the people, it's the bitter, deep-seated racism of Maycomb that the First Purchase congregation resists. For the building, it's the forces of nature and time that the building resists. It's still standing. So are the people.
The theme of innocence in this novel usually focuses on Scout and her initial lack of knowledge about the world. You can also consider Tom Robinson's literal innocence in that he hasn't committed the crime he's accused of. If we're looking for evidence that the congregation at First Purchase is "innocent" in the sense that they're ignorant about the realities of the world, like Scout, or that they're somehow all perfect or free from sin, we won't find it.
If we're looking for evidence that they're "innocent" in that their devotion to their faith is true and pure, that's something we will find. The church doesn't have any prayer books or hymnals. Tellingly, Scout calls these items the typical "impedimenta" found in the white church she usually attends, suggesting she thinks these items impede rather than aid worship. First Purchase doesn't have them, and indeed, they don't need them: the congregation is mostly illiterate, as we find out later in that same chapter. This congregation also does not need much decoration or even comfortable seating in their church. They're there to worship and come together as a community, not to admire the artwork or architecture or relax in luxury. The church is also damp and a bit cool inside, but the presence of the fans reveals that it will quickly become uncomfortably warm. The churchgoers' willingness to sit through this discomfort is evidence of their devotion, purity of faith, and commitment to their community. Whether you call this theme "innocence" is a matter of interpretation. I probably wouldn't.