In order to answer this question, we must look at the context in which the poem was written. Maya Angelou spent some of the most formative years of her childhood in the racially segregated town of Stamps, Arkansas. This experience, as well as her experiences with segregation and class-ism in St. Louis and California respectively, greatly colored the tone of her poetry. This poem, written in 1978, near the end of the Civil Rights movement, and at the height of the Black Power movement was meant to be a rallying cry for any persons who were victims of oppression, especially African Americans.
Initially one could attribute the "you" in this poem to refer to any person who supported segregation or racial injustice since the poem specifically references her African American heritage and the struggles of the African American people, in lines such as:
Bringing the gifts that my ancestors gave
I am the dream and the hope of the slave
However, Angelou was intentionally ambiguous by using the general "you," because the themes of rising above oppressors, self-love, and confidence in who you are despite your circumstances, transcend race, and apply to gender, class, and religion as well.