What is the thesis of Cathy O'Neil's Weapons of Math Destruction?
Cathy O'Neil used to work at a hedge fund, in risk analysis, and as a data scientist, in all of which occupations she used Big Data algorithms constantly. She appreciated the convenience of being able to sort huge amounts of complex data in seconds but increasingly began to question whether...
Unlock This Answer Now
Start your 48-hour free trial and get ahead in class. Boost your grades with access to expert answers and top-tier study guides. Thousands of students are already mastering their assignments—don't miss out. Cancel anytime.
Already a member? Log in here.
the models she used, or mathematical models in general, were actually effective.
This led her to research and write Weapons of Math Destruction. O'Neil concludes that while there are legitimate uses for statistical modeling, many of these models produce inaccurate results which cause harm to people and reinforce structural inequality in society. She refers to these models as WMDs, weapons of math destruction. A WMD has three principal characteristics:
1. It is opaque or invisible. This means that the people whose data is analyzed do not know how the model works or perhaps even that it exists at all.
2. It is unfair, damaging the lives of those whose data is being analyzed.
3. It is scalable, having the capacity to grow exponentially.
O'Neil points out that in the upper echelons of society, people are treated as individuals. Elite schools and employers conduct face-to-face interviews and take individual circumstances into account. Poor people, however, are processed en masse using mathematical models.
Therefore, O'Neil's thesis is that Big Data's WMDs, which can turn lives upside down by refusing credit, employment, or education, are not only inherently flawed and unfair, but biased towards the advantaged.
What is the thesis of Cathy O'Neil's Weapons of Math Destruction?
Cathy O'Neil is an academic mathematician who has also worked in the big data and finance industries. Weapons of Math Destruction is a critique of the way in which algorithms are used in a wide variety of fields, including finance, insurance, policing, and education, to make decisions which crucially affect people's lives. The people affected by these decisions generally do not know that the algorithms are being used, and even the organizations which use them do not know how they achieve their results, meaning that these decisions are never questioned.
The main argument in Weapons of Math Destruction is that the use of impersonal, opaque algorithms reinforces structural inequality in society in a way that may well be more intractable than personal prejudice. Fifty or sixty years ago, a person applying for a mortgage would have done so in person, probably speaking to a bank employee they already knew. This employee could have been racist and less willing to grant mortgages to Black families. However, what happens now is that an algorithm makes the decision, taking into account many factors which may well be racially specific, such as one's zip code. This decision is not seen as racist, since it is not personal, but it is rigid, unappealable, and discriminates against people of color. The same problem applies in the criminal justice system, college admissions, and many other areas.
What is the concept of scale in Cathy O'Neil's Weapons of Math Destruction?
In Cathy O'Neil's thought-provoking book on data and algorithms, she argues that there are three main elements of a WMD or weapons of math destruction. The three elements are opacity, scale, and damage. We won't get into opacity and damage right now. Let's just stick with scale, as that's what you asked about.
According to O'Neil, "Scale is what turns WMDs from local nuisances into tsunami forces." What leads O'Neil to make such a dire claim? To answer that, we must understand what she means by scale. She tells us what she means when she writes about a statistician wondering if their mathematical model/algorithm can "scale" or "grow exponentially."
We might want to think of scale like money. The more money you have, the better off you generally are. Same goes for algorithms. The more data you have, the better off you are. That's why scale is so important.
As O'Neil tells us, scaling has big problems. Algorithms can scale to such an extent that they are, according to O'Neil, "quickly establishing broad norms that exert upon us something very close to the power of the law."
Using examples of banks and prisons, O'Neil tells us about the harsh consequences of scale.
If a bank's model says you’re too high-risk for them to lend you money then, as O'Neil says, "you're a deadbeat."
Likewise, if a prison's algorithm determines that you're likely to commit another crime if you're released from jail, then you won't be set free anytime soon.
While a mathematician or statistician might benefit from scale, it appears as if a substantial amount of other people might suffer due to scale.