A broad-coverage challenge corpus for sentence understanding through inference

Adina Williams, Nikita Nangia, Samuel R. Bowman

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Abstract

    This paper introduces the Multi-Genre Natural Language Inference (MultiNLI) corpus, a dataset designed for use in the development and evaluation of machine learning models for sentence understanding. At 433k examples, this resource is one of the largest corpora available for natural language inference (a.k.a. recognizing textual entailment), improving upon available resources in both its coverage and difficulty. MultiNLI accomplishes this by offering data from ten distinct genres of written and spoken English, making it possible to evaluate systems on nearly the full complexity of the language, while supplying an explicit setting for evaluating cross-genre domain adaptation. In addition, an evaluation using existing machine learning models designed for the Stanford NLI corpus shows that it represents a substantially more difficult task than does that corpus, despite the two showing similar levels of inter-Annotator agreement.

    Original languageEnglish (US)
    Title of host publicationLong Papers
    PublisherAssociation for Computational Linguistics (ACL)
    Pages1112-1122
    Number of pages11
    ISBN (Electronic)9781948087278
    StatePublished - 2018
    Event2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2018 - New Orleans, United States
    Duration: Jun 1 2018Jun 6 2018

    Publication series

    NameNAACL HLT 2018 - 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies - Proceedings of the Conference
    Volume1

    Conference

    Conference2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2018
    CountryUnited States
    CityNew Orleans
    Period6/1/186/6/18

    ASJC Scopus subject areas

    • Linguistics and Language
    • Language and Linguistics
    • Computer Science Applications

    Fingerprint Dive into the research topics of 'A broad-coverage challenge corpus for sentence understanding through inference'. Together they form a unique fingerprint.

  • Cite this

    Williams, A., Nangia, N., & Bowman, S. R. (2018). A broad-coverage challenge corpus for sentence understanding through inference. In Long Papers (pp. 1112-1122). (NAACL HLT 2018 - 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies - Proceedings of the Conference; Vol. 1). Association for Computational Linguistics (ACL).