AI Has A Bias Problem, And It's Our Fault

In 2016, researchers from Boston University and Microsoft were working on artificial intelligence algorithms when they discovered racist and sexist tendencies in the technology underlying some of the most popular and critical services we use every day.

- Ben Dickson

The revelation went against the conventional wisdom that artificial intelligence doesn’t suffer from the gender, racial, and cultural prejudices that we humans do.

The researchers made this discovery while studying word-embedding algorithms, a type of AI that finds correlations and associations among different words by analyzing large bodies of text. For instance, a trained word-embedding algorithm can understand that words for flowers are closely related to pleasant feelings. On a more practical level, word embedding understands that the term “computer programming” is closely related to “C++,” “JavaScript” and “object-oriented analysis and design.” When integrated in a resume-scanning application, this functionality lets employers find qualified candidates with less effort. In search engines, it can provide better results by bringing up content that’s semantically related to the search term.

The BU and Microsoft researchers found that the word-embedding algorithms had problematic biases, though—such as associating “computer programmer” with male pronouns and “homemaker” with female ones. Their findings, which they published in a research paper aptly titled “Man is to Computer Programmer as Woman is to Homemaker?” was one of several reports to debunk the myth of AI neutrality and to shed light on algorithmic bias, a phenomenon that is reaching critical dimensions as algorithms become increasingly involved in our everyday decisions.

THE ORIGINS OF ALGORITHMIC BIAS