Yes, we should. Instead of focusing on affirmative action programs, we should do things that will create more opportunities for all poor Americans. The place to start would be with public schools. The best way to help poor people (African Americans among them) is to ensure that they get a good education. Once we ensure that they can get a good K-12 education, we must then ensure that they can go to college or obtain other training.
Most of the problems faced by African Americans today are not the product of discrimination in our own time. If poor African Americans were to get the same kind of education that middle-class whites get, their life chances would be greatly improved because there is not systemic racism today that would prevent them from going to college and/or getting good jobs.
If we would focus on creating better educational opportunities, we would help poor African Americans without creating the kind of racial animosity that affirmative action programs bring about.