Last Updated September 5, 2023.
The most significant quotes from the book explain why the current state of Big Data threatens our society and harms the individual. Throughout, O’Neil explains how the goals of Big Data are not always bad, but that the methods by which they are carried out have unintended consequences.
All of those sound like worthy goals, to be sure, but every ranking system can be gamed.
This quote comes from the chapter in which O’Neal explains how college rankings have impacted student behavior as well as the operations of the colleges themselves. In an attempt to revamp the way colleges are ranked, a new system proposed during the Obama administration threatened to remove funding if colleges didn’t meet certain criteria that would positively impact students, such as graduation rate. However, O’Neil reveals that all such systems can be easily gamed. What this means is that rankings encourage people to cheat in order to gain an advantage over others. This is even worse in systems in which rankings are based on proxies that are difficult to control. An environment like this that rewards dishonesty leads to dishonest rankings, an ethical dilemma that Big Data has yet to solve.
If the goal of testing is to find someone to blame, and to intimidate workers, then, as we’ve seen, a WMD that spews out meaningless scores gets an A-plus.
This quote comes at the end of O’Neal’s chapter on how Big Data affects our professional lives. After discussing scheduling and efficiency/profit maximizing goals, O’Neil criticizes value-added systems of evaluation, which are most commonly used in teacher evaluations. This quote sums up O’Neil’s claim that numbers without justification or transparency are arbitrary and should not be used in decisions regarding employment. If evaluations are used to communicate to workers the standards they must meet in order to maintain their jobs, and those evaluations are based on meaningless data, then O’Neil asserts that they are nothing more than an intimidation tactic used to threaten an individual’s sense of job security.
If we back away from them and treat mathematical models as a neutral and inevitable force, like the weather or the tides, then we abdicate our responsibility.
This quote comes from the final paragraph of the book’s conclusion. It summarizes the message O’Neil wants to communicate to her readers about the dangers of Big Data left unchecked. The reason that Big Data has been able to seize such control over so many facets of our lives is because of the public’s oblivious indifference. For O’Neil, continuing this apathetic sentiment is dangerous to the fabric of society as it should be: fair and beneficial. Without public oversight and legal accountability, Big Data will continue to run amok, damaging individual lives, the economy, and our democracy.