There are several lines in the poem "The Listeners" that give readers a general idea of what time of day it is. However, if you are looking for an exact hour, the poem does not give that accurate of a time frame. The most specific time that I can give is that the poem takes place sometime during the night. In line two of the poem, the narrator tells his audience that the Traveler is knocking on the moonlit door. An argument can be made that the moon is sometimes visible during the day; however, sunlight overpowers the moonlight during the day. The poem specifically mentions several times that it is the moon lighting up parts of the house.
Knocking on the moonlit door;
Then a bit later the following lines appear.
Stood listening in the quiet of the moonlight
To that voice from the world of men:
Stood thronging the faint moonbeams on the dark stair,
These lines further detail the fact that the poem takes place at night. It mentions more moonlight and dark stairs.