I would think the most important facts concerning the speaker or speakers of the poem (for we are never told if this is one voice or a collective voice of a group) is the kind of lifestyle that the speaker leads. Note how there are repeated references to the kind of activities the speaker engages in and the kind of standing he or she has in society. It is a life that is not one of educational achievement, to put it mildly, as the speaker says "We / Left school." The passtimes of the speaker involve "lurking late," "striking straight" (presumably referring to the speaker's prowess in pool), and "thinning gin." Although there is a confusing reference to "Singing sin" we can infer that this means that the speaker is a n'er-do-well who is not involved in society and just sits on the edge of it, only taking and not contributing to it. Of course, the devastating result of all of these negative practices becomes clear at the end of the poem:
Thus, when we think about what we are told about the speaker of this poem, the focus is on the negative practices he engages in.