Communications of the ACM
Technical Perspective: The Impact of Auditing for Algorithmic Bias
If you read news articles on the ethics of AI, you will repeatedly see the phrase "algorithmic bias" popping up. It refers to algorithms producing results that appear racist or sexist or displaying other forms of unfair bias. For example, when Amazon built a machine-learning model to score job applications, trained on historical hiring data at the company, it discovered the system downgraded female applicants—or anyone who mentioned activities associated with women (coach of women's soccer league, for example).a In a ground-breaking paper, researchers Joy Buolamwini and Timnit Gebrub documented substantial discrepancies across skin types and the reliability with which three commercial face classifiers could classify gender in facial images: these classifiers could identify gender with high (99%) accuracy for white men but accuracy for darker-skinned men and all women was lower, with errors as high as 35% on darker-skinned women.
The term "algorithmic bias" is used to refer to these unwarranted or unfair differential results on different groups. Of course, the algorithm itself is not biased, in the sense it is a mathematical object with no views about the world or fairness. The bias is something that we humans attribute to how the algorithm and its model function. Often, the unfair treatment is a consequence of training the model on biased data chosen or generated by humans. The data used to train the Amazon job application classifier was drawn from the history of Amazon hiring decisions, which apparently included a bias against women. The model trained on that data recognized a pattern in human behavior and reproduced—or maybe even amplified—it.c What Buolamwini and Gebru's groundbreaking 2018 "Gender Shades" study drew attention to was the importance of representative training data for facial recognition classifiers. They demonstrated this by conducting an algorithmic audit, testing the performance of classifiers on a benchmark dataset (the Pilot Parliaments Benchmark they constructed) with explicit attention to representation across classes of skin color and gender.
No entries found