This poem does not actually explicitly say that all people are "brothers" but that is what it is saying. It is saying that no people are foreign because all of them are our "brothers" -- they are just like us.
In the first stanza of the poem, Kirkup says that all other people are like us because they walk upon the Earth just like us. He says that they are just like us because they will die just as we will die.
In the second stanza, he says that they are just like us because they need to get in the harvest so that they can eat. They have needs that are the same as our needs. When wars happen, these needs are not met and they suffer just as we would suffer.
In the third stanza, Kirkup talks about people having the same need for love and living lives that are all essentially the same. That gives us at least four ways in which he says we are all the same -- things we all have in common.
In James Kirkup's poem "No Men Are Foreign" he writes that "attacking other humans is attacking our brothers." So, while he does not specifically refer to anyone directly, his statement implies that all humans are our brothers. The two things that he notes as being common to all are " eyes, like ours, that wake or sleep" and " strength that can be won by love."