In Cathy O'Neil's thought-provoking book on data and algorithms, she argues that there are three main elements of a WMD or weapons of math destruction. The three elements are opacity, scale, and damage. We won't get into opacity and damage right now. Let's just stick with scale, as that's what you asked about.

According to O'Neil, "Scale is what turns WMDs from local nuisances into tsunami forces." What leads O'Neil to make such a dire claim? To answer that, we must understand what she means by scale. She tells us what she means when she writes about a statistician wondering if their mathematical model/algorithm can "scale" or "grow exponentially."

We might want to think of scale like money. The more money you have, the better off you generally are. Same goes for algorithms. The more data you have, the better off you are. That's why scale is so important.

As O'Neil tells us, scaling has big problems. Algorithms can scale to such an extent that they are, according to O'Neil, "quickly establishing broad norms that exert upon us something very close to the power of the law."

Using examples of banks and prisons, O'Neil tells us about the harsh consequences of scale.

If a bank's model says you’re too high-risk for them to lend you money then, as O'Neil says, "you're a deadbeat."

Likewise, if a prison's algorithm determines that you're likely to commit another crime if you're released from jail, then you won't be set free anytime soon.

While a mathematician or statistician might benefit from scale, it appears as if a substantial amount of other people might suffer due to scale.