Last Reviewed on April 2, 2020, by eNotes Editorial. Word Count: 1263
Author Cathy O’Neil begins by writing about how she has loved mathematics ever since she was a child:
Math provided a neat refuge from the messiness of the real world.
She majored in math, obtained her PhD, and became a professor. Later, however, she left academia to work for a hedge fund. There, the 2008 financial crash showed her that mathematics “was not only deeply entangled in the world’s problems but also fueling many of them,” such as the housing crisis, unemployment, and financial collapse. However, financial institutions did not learn from the crash, and mathematics became more influential than ever with the rise of the Big Data economy.
O’Neil saw the convenience of Big Data, which could sort thousands of loan applications or other numerically-based documents in seconds, but was also aware of the problems, including a tendency to discriminate against those who were already poor. She refers to these Big Data models as WMDs: Weapons of Math Destruction. She uses an example of the Washington, DC, school district, which used a program to identify and fire underperforming teachers. However, this assessment method was seriously flawed, as it relied on a data sample of only 25–30 students and had no feedback mechanism. 206 teachers were fired, but the district will never know whether or not this decision was correct. The teachers are now viewed as failures, purely because the system had identified them as such—an example of what O’Neil calls “a WMD feedback loop.”
In the upper echelons of society, people tend to be personally evaluated. White-shoe law firms and exclusive preparatory schools conduct face-to-face interviews. The poor, however, are processed en masse by WMDs. The score that results “can turn someone’s life upside down,” even though it is only based on a probability, not on a certainty. However, people generally cannot fight back—and when they do, O’Neil notes,
The evidence must be ironclad. The human victims of WMDs . . . are held to a far higher standard of evidence than the algorithms themselves.
Having seen the danger posed by WMDs, O’Neil left the hedge fund where she worked in 2011 and became a data scientist. Seeing interviews with the Occupy Wall Street protesters made her realize that, although she agreed with them, they did not understand how finance worked. She decided to try to help by bringing the knowledge and information she had gained at the hedge fund to the cause of financial reform. She wanted to do this because she believes that the people who run WMDs, and even other data scientists, often fail to consider the people “on the receiving end of the transaction,” who are regarded as “collateral damage” (if anyone thinks about them at all).
The book, O’Neil writes, explores examples of people being harmed by WMDs “at critical life moments: going to college, borrowing money, getting sentenced to prison, or finding and holding a job.” This is the dark side of Big Data.
Chapter 1: Bomb Parts: What Is a Model?
O’Neil begins by observing how the managers of baseball teams organize their defenses based on analyses of opposing players’ hitting patterns. This is only one of the ways in which teams use statistical data to maximize the probability that they will win. The use of statistical models in baseball is fair, because everyone has access to the statistics and the data is highly relevant to the outcomes.
While the second point may sound obvious, the people who design WMDs, in contrast, “routinely lack data for the behaviors they’re most interested in.” This leads them to use proxies which do not necessarily correlate, and are sometimes discriminatory or even illegal, such as judging a subject’s ability to pay back a loan based on the...
(The entire section contains 1263 words.)
Unlock This Study Guide Now
Start your 48-hour free trial to unlock this Weapons of Math Destruction study guide. You'll get access to all of the Weapons of Math Destruction content, as well as access to more than 30,000 additional guides and more than 350,000 Homework Help questions answered by our experts.
- Chapter Summaries