Yamada starts by listing times in history where people do take individual action, but to no real result: bombings, protests, even fleeing to Canada. Take her mentioning of Kitty Genovese for example, a woman who DID take as much action as she could (she screamed the entire time she was murdered, and even though everyone watched, NO ONE helped). In Kitty's case, the sociology phenomenon of "diffusion of responsibility" occured, where everyone expected everyone else to take action, and thus no action was taken. That was the opposite of individual responsibility, in its worse-case scenario. She is saying that individual responsibility is a 2-way road: If I express outrage, you should respond similarly and help. But it doesn't work that way. Outrage is expressed, and no one takes responsibility for the evil. Take her mentioning of the 6 million futile letters to Congress and tatooing of stars on behalf of the Holocaust. The people still died, even though outrage was expressed.
Instead, the evils happen, and "You let'm/I let'm/All are punished", meaning, we all let these things happen, and we are all punished because of it.