The major downside to basing affirmative action on income rather than on race is the idea that (in some people’s minds, at least) race still matters. In other words, people who are middle class but are not white can, arguably, still face discrimination based on their race. Income-based affirmative action would not alleviate this problem.
The most famous study showing this problem is a study in which identical resumes are sent out to companies. Some resumes have “white” names like Emily Walsh while other resumes have “black” names like Lakisha Washington. The resumes with the white names consistently get more calls for interviews than those with black names. This sort of evidence tends to show that income-based affirmative action would not be enough because minority groups still face discrimination based not on income but on race.
This is an interesting question, as affirmative action usually helps minority groups that have been discriminated against in the past through education, business, and employment.
If we changed this to target people in poverty, this makes sense. We would be helping people who need it most, as not all people of a minority group needs help.
Based on this point, making affirmative action about the helping of people in poverty might seem like a perfectly good idea. However, there is one negative point.
We might overlook to help a minority group that still needs help. Moreover, if affirmative action is about compensation (which it is, to some extent), we might fail to compensate well enough.