In short, the answer is that religion is both a positive and a negative factor in Baldwin's story.
The church is a home and a refuge for the young John and others. He bonds with Elisha through the church, and this is one of the few things that give John a sense of fulfillment. John's father Gabriel (who is not his biological father) is a minister whose life in the church is the one source of empowerment and escape from the brutal oppression to which he's been subjected by a racial society, both in the South and the North.
Paradoxically, however, religion itself is another form of oppression. Though Christian belief is the subtext of the entire novel, Baldwin's theme is an awareness that religion has limited or even destroyed the freedom of African Americans. Gabriel rationalizes, through religion, his self-righteous and angry demeanor, and his condemnation of other people's alleged sins and waywardness, as he see it, is justified in his mind by the belief that he's carrying out God's work. His sister, Florence, finally confronts him on this, telling him that his perpetually judgmental stance toward John and Elizabeth is his way of deflecting from his own sins. Religion ironically is both a source and a cover for Gabriel's abusiveness. Gabriel refuses to acknowledge this, telling Florence she is going to be "cut down."
Baldwin himself was a free thinker in his adult life. As with other artists, religion and its language and imagery were part of his life and work without his being a "believer." During his meeting with Elijah Mohammed, Baldwin indicated, in response to Mohammed's request that he think about embracing Islam, that he was not interested in religion. Yet Baldwin's feelings are a microcosm of the modern intellectual mindset that draws on the legacy of religion while simultaneously regarding it as a superseded belief system of human thought.