Lines 70-72 run as follows:
Shall I say, I have gone at dusk through narrow streets
And watched the smoke that rises from the pipes
Of lonely men in shirt-sleeves, leaning out of windows?
The setting and the people here appear distinctly working class, while Prufrock and his milieu are middle class. Also, the people referred to here are male, whereas in his day-to-day life Prufrock appears to be surrounded by largely uncongenial society women. Not that we are ever told anything directly in this poem, of course, but we can deduce a good deal about Prufrock's situation from his rambling, disconnected thoughts as they are presented.
Prufrock talks vaguely and variously of making tea-time visits, with toast and marmalade, of dressing for dinner and guests making small talk about art – 'In the room the women come and go/Talking of Michelangelo’. All of this smacks unmistakably of polite, middle-class culture with Prufrock in the midst of it seemingly bored to death by such stifling routines, formalities and trivialities – ‘I have measured out my life in coffee spoons,’ he observes sardonically.
Prufrock talks of visiting – perhaps only in his imagination – worlds different to his own, of dirty dusty streets in the poorer part of town, where he imagines men as lonely as himself, sitting at the windows of their shabby little houses, smoking pipes and dressed in their shirts, with no coats. The middle-class Prufrock would not be able to dress so informally – he has to worry endlessly about donning morning coats, collars and so on – nor would he probably be able to indulge in something as unrefined as pipe-smoking. Therefore these men, although they might be poor, appear considerably less restricted than the unhappy Prufrock in some ways. He envisages a certain bond with them in his own loneliness, but he is prevented by class barriers from seeking them out as companions, as he would probably like to do.