First, Larkin uses an interesting pair of words in the title to convey a double meaning. "Church Going" explains the speaker's physical act of visiting this church and describing the scene he finds there. It also conveys that the church is going; it seems to be disappearing from importance in society.
With the exception of the second stanza, no stanzas end in a completed thought. In effect, this creates a long sense of inner dialogue as the speaker is torn between what purpose the church serves and the fate it seems doomed to suffer. This is echoed in the only stanza which does end in a complete thought, Stanza 2, where he notes that "the place was not worth stopping for." He notes the contradiction in the very next stanza:
Yet stop I did: in fact I often do,
And always end much at a loss like this,
Wondering what to look for; wondering, too,
When churches fall completely out of use
What we shall turn them into . . .
All other stanzas become a flow of thought, running over into each other just as his reasoning seems to do.
Repetition is used in the first line of the final stanza:
A serious house on serious earth it is
This conveys the overall tone and the importance the speaker places on the purpose the church has always served. In the same stanza, he notes that
Since someone will forever be surprising
A hunger in himself to be more serious
Those longing for wisdom have always gravitated toward the church, and the speaker conveys its importance in his final thoughts through this repetition.