The direct context of the quote is that the retired Woodifield, who has had a stroke, is only allowed out to visit the City--central London--once a week by his wife and daughters. They can't imagine what he does there, but then the narrator explains that "we cling to our last pleasures as the tree clings to its last leaves."
A tree clinging to its last leaves is an image of a tree near to death. This statement implies that Woodifield too is nearing death. He is an older man, and it does give him pleasure to get out and visit an old friend, as well as smoke a cigar.
By more than simply a reference to Woodifield, it sets the tone for a very bleak story which primarily concerns death: the death of Woodifield and the boss's sons in World War I (Woodifield has come to report that his daughters saw both graves). It provides a context and becomes a metaphor for the disturbing high point of the story, in which the boss deals with his grief over his dead son by torturing a fly to death, perhaps enacting how he feels he is slowly being tortured to death, drop by drop, by a malevolent god--or how he feels his son died, tortured on the battlefield, drop by drop. In the end, however, the boss does not derive "pleasure" from playing god, for
such a grinding feeling of wretchedness seized him that he felt positively frightened.
Mansfield shows us in this story a life with few pleasures as the older generation grapples with the destruction World War I has wrought.