FairPrep: Promoting data to a first-class citizen in studies on fairness-enhancing interventions

Sebastian Schelter, Yuxuan He, Jatin Khilnani, Julia Stoyanovich

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Abstract

    The importance of incorporating ethics and legal compliance into machine-assisted decision-making is broadly recognized. Further, several lines of recent work have argued that critical opportunities for improving data quality and representativeness, controlling for bias, and allowing humans to oversee and impact computational processes are missed if we do not consider the lifecycle stages upstream from model training and deployment. Yet, very little has been done to date to provide system-level support to data scientists who wish to develop responsible machine learning methods. We aim to fill this gap and present FairPrep, a design and evaluation framework for fairness-enhancing interventions, which helps data scientists follow best practices in ML experimentation. We identify shortcomings in existing empirical studies for analyzing fairness-enhancing interventions and show how FairPrep can be used to measure their impact. Our results suggest that the high variability of the outcomes of fairness-enhancing interventions observed in previous studies is often an artifact of a lack of hyperparameter tuning, and that the choice of a data cleaning method can impact the effectiveness of fairness-enhancing interventions.

    Original languageEnglish (US)
    Title of host publicationAdvances in Database Technology - EDBT 2020
    Subtitle of host publication23rd International Conference on Extending Database Technology, Proceedings
    EditorsAngela Bonifati, Yongluan Zhou, Marcos Antonio Vaz Salles, Alexander Bohm, Dan Olteanu, George Fletcher, Arijit Khan, Bin Yang
    PublisherOpenProceedings.org
    Pages395-398
    Number of pages4
    ISBN (Electronic)9783893180837
    DOIs
    StatePublished - 2020
    Event23rd International Conference on Extending Database Technology, EDBT 2020 - Copenhagen, Denmark
    Duration: Mar 30 2020Apr 2 2020

    Publication series

    NameAdvances in Database Technology - EDBT
    Volume2020-March
    ISSN (Electronic)2367-2005

    Conference

    Conference23rd International Conference on Extending Database Technology, EDBT 2020
    CountryDenmark
    CityCopenhagen
    Period3/30/204/2/20

    ASJC Scopus subject areas

    • Information Systems
    • Software
    • Computer Science Applications

    Fingerprint Dive into the research topics of 'FairPrep: Promoting data to a first-class citizen in studies on fairness-enhancing interventions'. Together they form a unique fingerprint.

  • Cite this

    Schelter, S., He, Y., Khilnani, J., & Stoyanovich, J. (2020). FairPrep: Promoting data to a first-class citizen in studies on fairness-enhancing interventions. In A. Bonifati, Y. Zhou, M. A. Vaz Salles, A. Bohm, D. Olteanu, G. Fletcher, A. Khan, & B. Yang (Eds.), Advances in Database Technology - EDBT 2020: 23rd International Conference on Extending Database Technology, Proceedings (pp. 395-398). (Advances in Database Technology - EDBT; Vol. 2020-March). OpenProceedings.org. https://doi.org/10.5441/002/edbt.2020.41