(QA)2: Question Answering with Questionable Assumptions

Najoung Kim, Phu Mon Htut, Samuel R. Bowman, Jackson Petty

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Abstract

    Naturally occurring information-seeking questions often contain questionable assumptions-assumptions that are false or unverifiable. Questions containing questionable assumptions are challenging because they require a distinct answer strategy that deviates from typical answers for information-seeking questions. For instance, the question When did Marie Curie discover Uranium? cannot be answered as a typical when question without addressing the false assumption Marie Curie discovered Uranium. In this work, we propose (QA)2 (Question Answering with Questionable Assumptions), an open-domain evaluation dataset consisting of naturally occurring search engine queries that may or may not contain questionable assumptions. To be successful on (QA)2, systems must be able to detect questionable assumptions and also be able to produce adequate responses for both typical information-seeking questions and ones with questionable assumptions. Through human rater acceptability on end-to-end QA with (QA)2, we find that current models do struggle with handling questionable assumptions, leaving substantial headroom for progress.

    Original languageEnglish (US)
    Title of host publicationLong Papers
    PublisherAssociation for Computational Linguistics (ACL)
    Pages8466-8487
    Number of pages22
    ISBN (Electronic)9781959429722
    StatePublished - 2023
    Event61st Annual Meeting of the Association for Computational Linguistics, ACL 2023 - Toronto, Canada
    Duration: Jul 9 2023Jul 14 2023

    Publication series

    NameProceedings of the Annual Meeting of the Association for Computational Linguistics
    Volume1
    ISSN (Print)0736-587X

    Conference

    Conference61st Annual Meeting of the Association for Computational Linguistics, ACL 2023
    Country/TerritoryCanada
    CityToronto
    Period7/9/237/14/23

    ASJC Scopus subject areas

    • Computer Science Applications
    • Linguistics and Language
    • Language and Linguistics

    Fingerprint

    Dive into the research topics of '(QA)2: Question Answering with Questionable Assumptions'. Together they form a unique fingerprint.

    Cite this