First Purchase African M. E. Church is Calpurnia's church, and one Sunday while Atticus is away, Jem and Scout attend a service with her. The reason why having books or hymnals at this church wouldn't have made a difference is that the majority of the congregants cannot read.
One of the first things Scout notices after entering the church is the lack of "ecclesiastical impedimenta" that she is accustomed to seeing in church. As part of this observation, she notices that there are no hymn books or church programs. She tries to ask Calpurnia about this just as the service is about to start, but Calpurnia shushes her. As Reverend Sykes announces that the congregation will "sing number two seventy-three," her confusion reaches fever pitch, but it is only later that Cal explains why hymn books would have made no difference at this church. For now, Scout must be content with watching as Reverend Sykes sings a line and has the congregation echo it back to him.
After the service, Jem asks Calpurnia why hymns are sung in a fashion so strange to him. Scout and Jem are surprised to learn that the overwhelming majority of congregants at First Purchase are unable to read. Given that the congregation is largely illiterate, having books would have been a waste of space and resources.