Central to understanding this poem is appreciating the deep irony that is evident in the reaction of the speaker to the theme he is trying to address. At surface level, at least, he is cynical and mocking about religion, churches and its importance to life. However, in this poem he explores his own mixed feelings and admits the way that churches and what they represent are something that he cannot so quickly shrug away as being unimportant to him.
Note how this irony is created. The first and second stanza present the church as nothing special: just another church with its "matting, seats and stone" and "sprawling flowers." Being alone in the church, the speaker feels free to say "Here endeth" as a kind of joke, and we are told that "The echoes snigger briefly," perhaps indicating the attitude of the speaker towards religion and churches.
However, in spite of his cynicism, the rest of the poem explores the speaker's rather complicated thoughts about church and its role in life. Even though he reflects that the church was not really worth stopping for, the next stanza begins with the statement "Yet stop I did," as he discusses the kind of magnetic attraction he feels that draws him to visit such churches. Even though he doesn't belive in the tenets of Christianity, he is forced to concede that churches fill some kind of whole or vacuum in our lives, as the last stanza makes clear:
A serious house on serious earth it is, In whose blent air all our compulsions meet, Are recognized, and robed as destinies. And that much never can be obsolete, Since someone will forever be surprising A hunger in himself to be more serious, And gravitating with it to this ground, Which, he once heard, was proper to grow wise in, If only that so many dead lie round.
Note how the speaker recognises that churches will never be "obsolete" because they satisfy the "hunger" within ourselves to be "more serious." Also spot the ironic closing line. Churches are good places to grow "wise in," if only for the fact that they are surrounded by graves. Thus the central irony of this excellent poem deals with the mixed feelings and emotions the speaker has regarding churches and religion in general.