A ploughman (in American English "plowman") is someone who directs a plow through a field by leading a horse or other animal attached to it. The exact line from Gray's "Elegy" is:

The ploughman homeward plods his weary way And leaves the world to darkness and to me.

Evening falls, and the narrator of the poem is left all alone in the church graveyard. The plowman has finished his work for the day and is now heading home for the night. Operating a plow can be extremely hard work, and if we didn't know that already, Gray helpfully reminds us by referring to the plowman plodding his "weary way" home. Now that the plowman has gone, and the narrator is alone, he can contemplate, among other things, the lives and fate of those ordinary country folk, like the plowman, buried beneath the old country churchyard.