CrowS-Pairs: A challenge dataset for measuring social biases in masked language models

Nikita Nangia, Clara Vania, Rasika Bhalerao, Samuel R. Bowman

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Abstract

    Warning: This paper contains explicit statements of offensive stereotypes and may be upsetting. Pretrained language models, especially masked language models (MLMs) have seen success across many NLP tasks. However, there is ample evidence that they use the cultural biases that are undoubtedly present in the corpora they are trained on, implicitly creating harm with biased representations. To measure some forms of social bias in language models against protected demographic groups in the US, we introduce the Crowdsourced Stereotype Pairs benchmark (CrowS-Pairs). CrowS-Pairs has 1508 examples that cover stereotypes dealing with nine types of bias, like race, religion, and age. In CrowS-Pairs a model is presented with two sentences: one that is more stereotyping and another that is less stereotyping. The data focuses on stereotypes about historically disadvantaged groups and contrasts them with advantaged groups. We find that all three of the widely-used MLMs we evaluate substantially favor sentences that express stereotypes in every category in CrowS-Pairs. As work on building less biased models advances, this dataset can be used as a benchmark to evaluate progress.

    Original languageEnglish (US)
    Title of host publicationEMNLP 2020 - 2020 Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference
    PublisherAssociation for Computational Linguistics (ACL)
    Pages1953-1967
    Number of pages15
    ISBN (Electronic)9781952148606
    StatePublished - 2020
    Event2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020 - Virtual, Online
    Duration: Nov 16 2020Nov 20 2020

    Publication series

    NameEMNLP 2020 - 2020 Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference

    Conference

    Conference2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020
    CityVirtual, Online
    Period11/16/2011/20/20

    ASJC Scopus subject areas

    • Information Systems
    • Computer Science Applications
    • Computational Theory and Mathematics

    Fingerprint

    Dive into the research topics of 'CrowS-Pairs: A challenge dataset for measuring social biases in masked language models'. Together they form a unique fingerprint.

    Cite this