Three notable elements in "Church Going" could be considered to include the rhyme scheme, the careful selection of vocabulary to create word-pictures in the mind of the reader, and the conscious effort to leave the message(s) of the poem open to interpretation by the reader.
The rhyme scheme seems calculated to make the reader feel slightly out-of-balance, as does the speaker while conducting his/her explorations of the church. The pattern of rhyme could be represented as "ababcadcd", very complicated and unexpected.
Descriptive phrases present impressions of the church building itself
matting, seats, and stone,
And little books; sprawlings of flowers, cut
For Sunday, brownish now; some brass and stuff
Up at the holy end; the small neat organ
and of the emotions the visit raises. The speaker "wonders" about what will happen to the building and how people will think of it in years to come, but yet "it pleases me to stand in silence here."
In the end, the speaker seems unsure of exactly what he feels about the place or why s/he stopped there, and the reader is left with that same question to be determined for him/herself.