In the first stanza, the boys are described as neglected and troubled.
The “rough” children do not seem to choose to be mean. They are clearly living in poverty, and have nearly no one to take care of them. They have “torn clothes” and are clearly skinny and poorly clothed.
Their thighs showed through rags. They ran in the street
And climbed cliffs and stripped by the country streams.
The boys he described “threw words like stones” not because they were evil, but because they were suffering. They had trouble putting their thoughts into words because they felt abandoned. The speaker wants to “forgive them” but he can’t because although he tries to laugh it off, they never smile back.
This poem describes the difficultly of youth. The speaker suffers from taunting, but the boys who taunt him are suffering too. The speaker makes sure we know that. Nothing is simple in life.