Weapons of Math Destruction

by Cathy O'Neil

Start Free Trial

Chapters 3–5 Summary

Download PDF PDF Page Citation Cite Share Link Share

Chapter 3: Arms Race: Going to College

O’Neil compares the operation of WMDs on a vast scale to the effects of everyone in the country eating exactly the same diet, which would cause a loss of diversity, along with hugely inflated prices for popular foods. She then says that when a formula that works on a small scale grows to become a national or global standard, “it creates its own distorted and dystopian economy.”

One example is the U.S. News and World Report ranking of universities and colleges. There is no single clear way to measure a value such as “educational excellence,” so the journalists measure things that can be quantified: SAT scores of students, acceptance rates, and the percentage of students who graduate. After the rankings became influential, however, they initiated a feedback loop. Colleges which fared badly would fail to attract talented students and professors, causing their rankings to decline further. University administrators are therefore forced to dedicate themselves to boosting their ranking in arbitrarily defined categories by manipulating the fifteen proxies selected by U.S. News instead of focusing on education. Some have even submitted false data to inflate their scores.

The factors U.S. News leaves out of their assessment have been as important as what they include. One is the cost of attending any given university. The rankings give universities no incentive to keep fees low; perhaps not coincidentally, the cost of going to college increased by 500% in the years between 1985 and 2013. The rankings cannot be held entirely responsible for this, but O’Neil argues that they have contributed to the problem and have encouraged universities to “manage their student populations almost like an investment portfolio.” Big Data has also allowed universities to select students who are predicted to do the most to maximize their future rankings. In this atmosphere, a culture of attempting to “game the system” has grown, along with an industry of coaches and tutors who study colleges’ admissions models closely. Poorer students are at a disadvantage, since their families cannot afford to spend tens of thousands of dollars on these services.

The Obama administration attempted to reform the ranking system, but failed. O’Neil says that this may be just as well, since any system can be gamed. Instead, the Department of Education released extensive data about colleges on its website, with the hopes that students can use this to make informed decisions without reference to rankings. O’Neil regards this transparent approach as “the opposite of a WMD.”

Chapter 4: Propaganda Machine: Online Advertising

While the U.S. News rankings have made life difficult for even wealthy students and their families, for-profit colleges (also known as “diploma mills”) target the poor “with the bait of upward mobility.” Mathematical modeling has made it easy for such predatory institutions to pinpoint those who are most vulnerable to their message. Recruiters find what is known as the “pain point” in each consumer, the area in which they suffer most and have least confidence. This might be low self-esteem, drug addiction, bereavement, a history of abuse, or any one of many other factors, often unwittingly disclosed in Google searches or questionnaires.

The internet provides the opportunity for data analysis on a scale that was previously unimaginable. The “quadrillions of words” produced by internet users have been particularly valuable in the development of natural-language machines and linguistic analysis. An advertising program can quickly build up a sophisticated picture of the people it targets and make relatively accurate predictions about their behavior.

For-profit colleges use WMDs to find and target the poorest students, exacerbating the wealth gap by taking...

(This entire section contains 1227 words.)

Unlock this Study Guide Now

Start your 48-hour free trial and get ahead in class. Boost your grades with access to expert answers and top-tier study guides. Thousands of students are already mastering their assignments—don't miss out. Cancel anytime.

Get 48 Hours Free Access

the little money they have, along with large amounts of taxpayer-funded financial aid, for diplomas which are often useless in the workplace. Meanwhile, the presidents of these diploma mills make millions of dollars a year, meaning that for-profit colleges essentially “drive inequality in both directions.”

There are many other industries apart from for-profit colleges that use predatory, WMD-driven advertising. One of the principal offenders is the payday loan industry, which offers short-term loans at an average interest rate of 574%. Quite apart from these unethical practices, the information consumers provide in filling out the applications for these loans is subject to abuse, including the possibility of theft. Regulators are now seeking to curb the worst excesses of the market in personal data with additional legislation.

Chapter 5: Civilian Casualties: Justice in the Age of Big Data

In 2013, the Police Chief of Reading, Pennsylvania, bought a crime prediction software program that “processed historical crime data and calculated, hour by hour, where crimes were most likely to occur.” A year later, he was able to announce that burglaries had decreased by 23%. Many other police departments throughout America have purchased similar programs.

One problem with these programs is that when the police spend more time in areas they highlight, they record the low-level “nuisance” crimes endemic to these neighborhoods, creating a feedback loop. The police generally include these crimes in statistics (and punish offenders) because they are following the “broken windows” theory of policing, which holds that low-level disorder must be strictly managed in order to prevent an escalation to more serious crimes. This is one of several models of policing. O’Neil contends that “mathematical models now dominate law enforcement. And some of them are WMDs.”

The models that police use focus on poor neighborhoods and therefore target poor people and ethnic minorities, even though the models are supposedly “color-blind.” Crimes committed by rich people, such as financial fraud, practically always go unpunished. O’Neil argues,

The result is that we criminalize poverty, believing all the while that our tools are not only scientific but fair.

The “stop and frisk” policy instituted by Mayor Michael Bloomberg in New York involved apparently random searches of people who appeared suspicious to police officers. About 85% of these encounters “involved young African American or Latino men.” Only one in a thousand was connected with a violent crime. However, many were charged with lesser crimes, such as underage drinking. Although this policy is not a WMD, it operates in a similar way: “stop and frisk” relies on the simple calculation that the more people police stop, the more crime they will uncover—even if these crimes are generally minor. The process also creates a feedback loop, whereby men from ethnic minorities are brought into the criminal justice system, creating more data to justify the searches. The police had effectively discarded the concept of probable cause.

O’Neil says that WMDs favor efficiency over fairness. She points out that the models used by Amazon to predict repeat purchases are far more sophisticated than those used by the criminal justice system to predict repeat offenses or for any other purpose. Prisons become “black boxes” that produce no data to challenge current models.

Facial recognition technology and other developments have also led to a dramatic increase in surveillance. This is likely to continue, along with predictive models which will suggest how likely people are to commit crimes in the future. However, O’Neil argues that this technology is already being used too aggressively by police departments that are simply focused on reducing the number of crimes rather than trying to uphold standards in communities. Because such qualities as trust are hard to quantify, WMDs lead police into numerically-based strategies that are ineffective in the long term.

Previous

Introduction and Chapters 1–2 Summary

Next

Chapters 6–8 Summary