The phrase "hoary-headed swain" means a graying or white haired rustic person. In other words, a "swain" is an older and humble individual who does farm work or rural labor for a living.

The speaker has gone on at length about the humble people who are buried in the country graveyard. They are simple folk who have never achieved fame or fortune in life but have instead lived and died in quiet obscurity. The narrator, looking at their graves, celebrates their lives. They are, to him, good people who faithfully did their duty every day without complaining or asking for more. They might have been born with the talent of a great poet, like Milton, or the skills of a great statesmen, but the circumstances of their lives meant they were not able to develop their talents. Nevertheless, the speaker asserts, their lives have worth.

As he wanders among their graves, wondering about them, the speaker addresses himself as "thee" or you. He asks what a "kindred spirit" wandering through the graves might think should that person see him, also wandering in the cemetery? He then states that he if he is lucky, the kindred spirit would ask an ordinary old man (a "hoary-headed swain") about him, and the swain would say that the speaker is often noticed in the early mornings in the dewy graveyard hurrying to watch the sunrise on the "upland lawn."