In childhood, the boy represents everything that James cannot be or have.
“I created an imaginary world for myself. I believed my true self was a boy who lived in the mirror. I’d lock myself in the bathroom and spend long hours playing with him. . . . I had an ache inside, a longing, but I didn’t know where it came from or why I had it. The boy in the mirror, he didn’t seem to have an ache.”
Thus James can take out all his anger & frustration on the boy in the mirror, who will listen without ever judging him. On the one hand, James needs the boy to reflect what he could have, but at the same time, he hates the boy because "he didn't have a white mother, he never went hungry, he didn't have to share his bed."
As an adult, James looks back on his imaginary friend as a symbol of what he has overcome. He had triumphed over the fristrations and injustices the imaginary boy represented.