It is important to remember that the significance of dusk is given to the story by Norman Gortsby, who sees in this time of day a symbolism that he finds morosely pleasing to his own state of mind and what he is thinking and feeling. Note how the scene of "wide emptiness" and "unconsidered figures" appeals to Gortsby:
The scene pleased Gortsby and harmonised with his present mood. Dusk, to his mind, was the hour of the defeated. Men and women, who had fought and lost, who hid their fallen fortunes and dead hopes as far as possible from the scrutiny of the curious, came forth in this hour of gloaming, when their shabby clothes and bowed shoulders and unhappy eyes might pass unnoticed, or, at any rate, unrecognised.
Thus the significance of dusk is entirely created by Gortsby, who is naturally drawn to such a time of day, which he associates with being the "hour of the defeated." He looks at this time through his own eyes and out of his own situation, which, as the story tells us, finds an echo with his own failure as Gortsby as well counts himself among the "defeated." Dusk then, to Gortsby's mind, is a time when those who have failed in life in whatever way can be free to come out without being noticed and talked about by others. Dusk provides such people with an anonymity that it is suggested they crave.