In "My Parents," Spender explores the theme of the isolation and alienation he felt from the other boys his age in his neighborhood. They were poor, rough boys dressed in torn, ragged clothing who played in the streets. His parents, he says in the opening line, "kept him" from such children—except that they didn't, not really.

After all, these strong, tough boys bullied Spender physically and frightened him. They also mocked him, imitating his lisp and pointing at him. The poem shows Spender's perception that is was "them" versus "him."

In the last two lines of the poem, Spender says that when the boys threw mud, he looked the other way, "pretending to smile." With the smile, he was trying to get along with them and make a connection that would allow him to feel less isolated and alienated from these lads his own age. However, even though he wanted to "forgive" them, they never smiled back, so there could be no connection. These last two lines therefore reinforce the poem's theme of isolation and alienation.

The last two lines express the isolation the speaker felt because his parents kept him separate and protected from "children who were rough." The children who surrounded him engaged in typical childhood roughhousing; they "ran in the street and climbed cliffs and stripped by the country streams." The speaker feared them both physically,when they roughed him up, and psychologically, as they mocked his speech impediment when they followed him.  

The last two lines, 

And I looked another way, pretending to smile,
I longed to forgive them, yet they never smiled.

suggest that instead of being kept away from these children, the speaker longed to live among them to experience the give and take of childhood relationships. He understands that he would have sometimes received the same rough treatment, but if they had been intimate, he expects that he would have also received apologies from them instead of remaining estranged and isolated.