When the speaker claims to be "the darker brother," it is ironic because one would likely not think to treat their brother in the way that he is treated by white people. He compares himself to someone who is treated like the black sheep of their family, someone of whom the family is ashamed, someone that the family attempts to hide. It's like he is sent to "eat in the kitchen / When company comes" because they do not want him to be seen. It would be difficult to continue to think of people who would treat a person in this way as that person's family. Family is accepting and loving, or, at least, it should be.

The speaker uses this metaphor to represent how he is treated as a black man by white America. He is treated like a second-class citizen, something to be embarrassed about, something to hide or suppress. It is ironic, then, that he would compare people who treat him in this demeaning and demoralizing way to a close family member. Irony is created when there is a discrepancy between what we might expect and what actually happens. I would expect a person treated in this way to think of his persecutors as enemies, as adversaries; and instead, this speaker paints his persecutors as something like family. This correlation is certainly unexpected to me.