I’m Reading: “Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy” by Cathy O’Neil

We are surrounded by algorithms. We are constantly being evaluated by criteria that is invisible to us.

What I see on Google is different to what you see. What I see on Facebook is different to your perspective and not just because I have access to different thing to you.

I pay for insurance for a number of things, the cost of that insurance is governed by a set of parameters that are unknown to me; many of which I can’t change or even validate whether they are correct.

Weapons of Math Destruction explores some of these algorithms and their impacts on individuals and society in general.

Statisticians have known that many statistics have a dark side creating unintended consequences and perverse outcomes. As we increasingly use data, and the associated statistical algorithms, we need to understand the dangers of the perverse outcomes that we are creating.

Cathy O’Neil uses examples to illustrate the challenges that we are facing. The bulk of the book is examples of Weapons of Math Destruction (WMD) that already exist. There are examples for algorithms being used for politics, employee candidate selection, criminal justice, insurance, education ratings and advertising, to name just a few. The extent of these algorithms means that it’s unlikely that you haven’t been impacted in some way by one of them, but how do you know that the assessment of you is fair, or even accurate. How do you know what parameters have been used to calculate your insurance premium?

In many of the areas outlined in the book the unintended consequences lead to significant mistreatment of individuals and whole people groups. Many of these people groups being the same people groups that have been mistreated by society for generations – the poor, those living in certain neighbourhoods, ethnic minorities and women being particularly negatively impacted.

The book talks about a lot of examples and raises a lot of questions and concerns, the book doesn’t spend a lot of time exploring the potential resolutions to the issues raised. There are a few great thoughts in that direction but it’s not that primary topic for this book.

I’m quite sure that we don’t, yet, have the necessary regulatory framework in place for these algorithms. I’m also convinced that we will make progress towards the right framework, but in the interim, damage is being caused.

I read this book in the middle of a political and media storm about an organisation called Cambridge Analytica who collected data from Facebook on 50 million people. This story was pioneered by The Guardian with a lot of coverage on 17th March 2018 quoting whistleblower Christopher Wylie, but it’s worth noting that Cathy O’Neil’s book was published in September 2016 and contains many of the same details about Cambridge Analytica that we now regard as shocking. Perhaps news doesn’t travel as quickly as we think it does.

I was first prompted to read this book by Cathy’s TED talk which will give you an idea of the WMD that she has collected: