Supervised discovery of unknown unknowns through test sample mining (student abstract)

Zheng Wang, Bruno Abrahao, Ece Kamar

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Abstract

    Given a fixed hypothesis space, defined to model class structure in a particular domain of application, unknown unknowns (u.u.s) are data examples that form classes in the feature space whose structure is not represented in a trained model. Accordingly, this leads to incorrect class prediction with high confidence, which represents one of the major sources of blind spots in machine learning. Our method seeks to reduce the structural mismatch between the training model and that of the target space in a supervised way. We illuminate further structure through cross-validation on a modified training model, set up to mine and trap u.u.s in a marginal training class, created from examples of a random sample of the test set. Contrary to previous approaches, our method simplifies the solution, as it does not rely on budgeted queries to an Oracle whose outcomes inform adjustments to training. In addition, our empirically results exhibit consistent performance improvements over baselines, on both synthetic and real-world data sets.

    Original languageEnglish (US)
    Title of host publicationAAAI 2020 - 34th AAAI Conference on Artificial Intelligence
    PublisherAAAI press
    Pages13959-13960
    Number of pages2
    ISBN (Electronic)9781577358350
    StatePublished - 2020
    Event34th AAAI Conference on Artificial Intelligence, AAAI 2020 - New York, United States
    Duration: Feb 7 2020Feb 12 2020

    Publication series

    NameAAAI 2020 - 34th AAAI Conference on Artificial Intelligence

    Conference

    Conference34th AAAI Conference on Artificial Intelligence, AAAI 2020
    Country/TerritoryUnited States
    CityNew York
    Period2/7/202/12/20

    ASJC Scopus subject areas

    • Artificial Intelligence

    Fingerprint

    Dive into the research topics of 'Supervised discovery of unknown unknowns through test sample mining (student abstract)'. Together they form a unique fingerprint.

    Cite this