"The Listeners" by Walter De la Mare is really not a poem with a "message." Walter de la Mare, in fact, would have considered works with "messages" simply bad poetry rather than genuine works of the imagination. "The Listeners" is a poem intended to evoke a mood and setting rather than convey some ideological point.
The protagonist of the poem arrives at a deserted house in the dark. The house appears to be in a remote, rural setting. He knocks on the door and calls out, but no one answers, and eventually he leaves. The "listeners" of the title do not appear to be living human beings. Perhaps they are ghosts, or perhaps they merely are memories. The point of the poem is the uncanny nature of the experience.
As readers, we are not meant to think of this as a puzzle to be solved, but rather to empathize with the sense of mystery and melancholy.