Saki describes the setting of "Dusk" from the point of view of a young man called Norman Gortsby.
Dusk, to his mind, was the hour of the defeated. Men and women, who had fought and lost, who hid their fallen fortunes and dead hopes as far as possible from the scrutiny of the curious, came forth in this hour of gloaming, when their shabby clothes and bowed shoulders and unhappy eyes might pass unnoticed, or, at any rate, unrecognized.
This strongly suggests that the defeated were defeated in the general struggle for survival and not by any particular persons. Saki is suggesting that life is bellum omnium contra omnes (the war of all against all). If, for example, a man like Norman Gortsby lands a good job, that means a number of other men are prevented from landing that job. There is no hand-to-hand fighting, but there are only so many good positions, and there must be losers as well as winners.
According to Magill's Critical Survey of Short Fiction, Second Revised Edition (see reference link below):
Born in colonial Burma (now Myanmar) to a family that had for generations helped to rule the British Empire, Hector Hugh Munro grew up in a Devonshire country house where, reared along with his brother and sister by two formidable aunts, he had the secluded and strictly supervised sort of childhood typical of the Victorian rural gentry.
Saki has also been described as a Tory, a reactionary, a misogynist and an anti-Semite. He obviously has little sympathy for "the defeated." He was most likely a "social Darwinist" who believed that the human race is improved by the survival of the fittest in the struggle for existence.
Gortsby is not really a successful person, nor is he a gentleman of leisure. He is just an intelligent young man with a better-than-average job, probably lingering on a park bench because he has been working in an office and wants to relax in the open air before going home to his modest lodging. He has to be someone to whom the loss of a sovereign (one pound) would be painful; otherwise the point would be lost. In those Victorian times the average salary for a clerk was a pound a week.
Gortsby begins by despising the defeated people he sees around him. Then when he finds the cake of soap and assumes it belongs to the young con man, he has a change of heart. He tells himself:
"It's a lesson to me not to be too clever in judging by circumstances."
But when he discovers that the cake of soap belonged to the elderly gentleman who had been sitting beside him, he realizes that his original attitude of skepticism and indifference was the correct one. Saki is suggesting that it is a mistake to feel sorry for people. As a reactionary, he was undoubtedly opposed to government measures to help the unfortunate. He would have found private charity acceptable but must have hated and feared the growing socialist movement. It was one thing to give willingly and out of sympathy, and quite another to pass laws taxing one person to give his money to someone else, someone who might or might not really need or might it.
There is a strong possibility that the elderly gentleman was also a con man. Why else would he be so sure that the cake of soap had to be near the bench? He may have left it there deliberaely, intending to come back and use it as a ploy to start a conversation with Gortsby. Saki seems to be suggesting that the world is full of people who want to live without working, and that many of them come out in the dusk.