In this satirical short story of Saki's, Norman Gortsby is an unsympathetic, cynical observer of the elderly gentleman "with a drooping air of defiance" who sits on the park bence by his side. As Gortsby observes, the gentleman's clothes are not exactly shabby, but he has the appearance of one who does not have any extra income for the purchase of frivolous articles such as a box of chocolates.
He belonged unmistakably to that forlorn orchestra to whose piping no one dances; he was one of the world's lamenters who induce no responsive weeping.
In other words, the older gentleman belongs to the social group of people to whom no one would bother paying attention or provide any sympathy. As an elderly man, he is on the fringe of society, no longer of use in the business world or social world. His sorrows are of no concern to anyone other than, perhaps, a close relative. As he departs, Gortsby imagines that he returns to a house where he is ignored and considered of no importance.