• 38 Citations

Abstract

Artificial intelligence and machine learning are in a period of astounding growth. However, there are concerns that these technologies may be used, either with or without intention, to perpetuate the prejudice and unfairness that unfortunately characterizes many human institutions. Here we show for the first time that human-like semantic biases result from the application of standard machine learning to ordinary language---the same sort of language humans are exposed to every day. We replicate a spectrum of standard human biases as exposed by the Implicit Association Test and other well-known psychological studies. We replicate these using a widely used, purely statistical machine-learning model---namely, the GloVe word embedding---trained on a corpus of text from the Web. Our results indicate that language itself contains recoverable and accurate imprints of our historic biases, whether these are morally neutral as towards insects or flowers, problematic as towards race or gender, or even simply veridical, reflecting the status quo for the distribution of gender with respect to careers or first names. These regularities are captured by machine learning along with the rest of semantics. In addition to our empirical findings concerning language, we also contribute new methods for evaluating bias in text, the Word Embedding Association Test (WEAT) and the Word Embedding Factual Association Test (WEFAT). Our results have implications not only for AI and machine learning, but also for the fields of psychology, sociology, and human ethics, since they raise the possibility that mere exposure to everyday language can account for the biases we replicate here.
LanguageEnglish
Pages183-186
Number of pages4
JournalScience
Volume356
Issue number6334
DOIs
StatusPublished - 14 Apr 2017

Fingerprint

Learning systems
Semantics
Artificial intelligence

Keywords

  • cs.AI
  • cs.CL
  • cs.CY
  • cs.LG

Cite this

Semantics derived automatically from language corpora necessarily contain human biases. / Caliskan, Aylin; Bryson, Joanna J; Narayanan, Arvind.

In: Science, Vol. 356, No. 6334, 14.04.2017, p. 183-186.

Research output: Contribution to journalArticle

Caliskan, Aylin ; Bryson, Joanna J ; Narayanan, Arvind. / Semantics derived automatically from language corpora necessarily contain human biases. In: Science. 2017 ; Vol. 356, No. 6334. pp. 183-186
@article{7c2d36d9579a45649fbfa622eade17a3,
title = "Semantics derived automatically from language corpora necessarily contain human biases",
abstract = "Artificial intelligence and machine learning are in a period of astounding growth. However, there are concerns that these technologies may be used, either with or without intention, to perpetuate the prejudice and unfairness that unfortunately characterizes many human institutions. Here we show for the first time that human-like semantic biases result from the application of standard machine learning to ordinary language---the same sort of language humans are exposed to every day. We replicate a spectrum of standard human biases as exposed by the Implicit Association Test and other well-known psychological studies. We replicate these using a widely used, purely statistical machine-learning model---namely, the GloVe word embedding---trained on a corpus of text from the Web. Our results indicate that language itself contains recoverable and accurate imprints of our historic biases, whether these are morally neutral as towards insects or flowers, problematic as towards race or gender, or even simply veridical, reflecting the status quo for the distribution of gender with respect to careers or first names. These regularities are captured by machine learning along with the rest of semantics. In addition to our empirical findings concerning language, we also contribute new methods for evaluating bias in text, the Word Embedding Association Test (WEAT) and the Word Embedding Factual Association Test (WEFAT). Our results have implications not only for AI and machine learning, but also for the fields of psychology, sociology, and human ethics, since they raise the possibility that mere exposure to everyday language can account for the biases we replicate here.",
keywords = "cs.AI, cs.CL, cs.CY, cs.LG",
author = "Aylin Caliskan and Bryson, {Joanna J} and Arvind Narayanan",
year = "2017",
month = "4",
day = "14",
doi = "10.1126/science.aal4230",
language = "English",
volume = "356",
pages = "183--186",
journal = "Science",
issn = "0036-8075",
publisher = "American Association for the Advancement of Science",
number = "6334",

}

TY - JOUR

T1 - Semantics derived automatically from language corpora necessarily contain human biases

AU - Caliskan,Aylin

AU - Bryson,Joanna J

AU - Narayanan,Arvind

PY - 2017/4/14

Y1 - 2017/4/14

N2 - Artificial intelligence and machine learning are in a period of astounding growth. However, there are concerns that these technologies may be used, either with or without intention, to perpetuate the prejudice and unfairness that unfortunately characterizes many human institutions. Here we show for the first time that human-like semantic biases result from the application of standard machine learning to ordinary language---the same sort of language humans are exposed to every day. We replicate a spectrum of standard human biases as exposed by the Implicit Association Test and other well-known psychological studies. We replicate these using a widely used, purely statistical machine-learning model---namely, the GloVe word embedding---trained on a corpus of text from the Web. Our results indicate that language itself contains recoverable and accurate imprints of our historic biases, whether these are morally neutral as towards insects or flowers, problematic as towards race or gender, or even simply veridical, reflecting the status quo for the distribution of gender with respect to careers or first names. These regularities are captured by machine learning along with the rest of semantics. In addition to our empirical findings concerning language, we also contribute new methods for evaluating bias in text, the Word Embedding Association Test (WEAT) and the Word Embedding Factual Association Test (WEFAT). Our results have implications not only for AI and machine learning, but also for the fields of psychology, sociology, and human ethics, since they raise the possibility that mere exposure to everyday language can account for the biases we replicate here.

AB - Artificial intelligence and machine learning are in a period of astounding growth. However, there are concerns that these technologies may be used, either with or without intention, to perpetuate the prejudice and unfairness that unfortunately characterizes many human institutions. Here we show for the first time that human-like semantic biases result from the application of standard machine learning to ordinary language---the same sort of language humans are exposed to every day. We replicate a spectrum of standard human biases as exposed by the Implicit Association Test and other well-known psychological studies. We replicate these using a widely used, purely statistical machine-learning model---namely, the GloVe word embedding---trained on a corpus of text from the Web. Our results indicate that language itself contains recoverable and accurate imprints of our historic biases, whether these are morally neutral as towards insects or flowers, problematic as towards race or gender, or even simply veridical, reflecting the status quo for the distribution of gender with respect to careers or first names. These regularities are captured by machine learning along with the rest of semantics. In addition to our empirical findings concerning language, we also contribute new methods for evaluating bias in text, the Word Embedding Association Test (WEAT) and the Word Embedding Factual Association Test (WEFAT). Our results have implications not only for AI and machine learning, but also for the fields of psychology, sociology, and human ethics, since they raise the possibility that mere exposure to everyday language can account for the biases we replicate here.

KW - cs.AI

KW - cs.CL

KW - cs.CY

KW - cs.LG

UR - https://doi.org/10.1126/science.aal4230

U2 - 10.1126/science.aal4230

DO - 10.1126/science.aal4230

M3 - Article

VL - 356

SP - 183

EP - 186

JO - Science

T2 - Science

JF - Science

SN - 0036-8075

IS - 6334

ER -