There is no way to answer this that is objectively provable and correct. However, I would argue that many Americans are touchy on this subject because both sides realize that their arguments are, in part, wrong. They do not like the fact that they cannot feel certain of being correct.
In the debate over affirmative action, neither side is completely correct. Looking at the pro-affirmative action side, it is very hard for them to deny that there is something fundamentally un-American about affirmative action. The idea of giving people preference due to their race or their sex seems very much opposed to what Americans are supposed to believe in. I would argue that proponents of affirmative action realize this and are made uneasy by supporting something that is in some ways antithetical to American values.
On the other hand, anti-affirmative action people have a hard time denying that affirmative action is in some ways justified. They know that African Americans and women have been discriminated against in the past. They know that there is no equal playing field for the races or the sexes today. Therefore, they feel conflicted about their opposition to affirmative action.
Thus, I would argue that both sides are “touchy” because they feel conflicted. They believe in their own opinions, but they also realize that there is something to be said for the other side and that realization makes them uncomfortable.