CrowS-Pairs: A Challenge Dataset for Measuring Social Biases
in Masked Language Models
Nikita NangiaClara VaniaRasika Bhalerao
CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models
Pretrained language models, especially masked language models (MLMs) have
seen success across many NLP tasks. However, there is ample evidence that they
use the cultural biases that are undoubtedly present in the corpora they are
trained on, implicitly creating harm with biased representations. To measure
some forms of social bias in language models against protected demographic
groups in the US, we introduce the Crowdsourced Stereotype Pairs benchmark
(CrowS-Pairs). CrowS-Pairs has 1508 examples that cover stereotypes dealing
with nine types of bias, like race, religion, and age. In CrowS-Pairs a model
is presented with two sentences: one that is more stereotyping and another that
is less stereotyping. The data focuses on stereotypes about historically
disadvantaged groups and contrasts them with advantaged groups. We find that
all three of the widely-used MLMs we evaluate substantially favor sentences
that express stereotypes in every category in CrowS-Pairs. As work on building
less biased models advances, this dataset can be used as a benchmark to
evaluate progress.
Nikita Nangia Clara Vania Rasika Bhalerao Samuel R. Bowman