Measuring fairness in ranked outputs

Ke Yang, Julia Stoyanovich

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Abstract

    Ranking and scoring are ubiquitous. We consider the setting in which an institution, called a ranker, evaluates a set of individuals based on demographic, behavioral or other characteristics. The final output is a ranking that represents the relative quality of the individuals. While automatic and therefore seemingly objective, rankers can, and often do, discriminate against individuals and systematically disadvantage members of protected groups. This warrants a careful study of the fairness of a ranking scheme, to enable data science for social good applications, among others. In this paper we propose fairness measures for ranked outputs. We develop a data generation procedure that allows us to systematically control the degree of unfairness in the output, and study the behavior of our measures on these datasets. We then apply our proposed measures to several real datasets, and detect cases of bias. Finally, we show preliminary results of incorporating our ranked fairness measures into an optimization framework, and show potential for improving fairness of ranked outputs while maintaining accuracy.

    Original languageEnglish (US)
    Title of host publicationSSDBM 2017
    Subtitle of host publication29th International Conference on Scientific and Statistical Database Management
    PublisherAssociation for Computing Machinery
    ISBN (Electronic)9781450352826
    DOIs
    StatePublished - Jun 27 2017
    Event29th International Conference on Scientific and Statistical Database Management, SSDBM 2017 - Chicago, United States
    Duration: Jun 27 2017Jun 29 2017

    Publication series

    NameACM International Conference Proceeding Series
    VolumePart F128636

    Other

    Other29th International Conference on Scientific and Statistical Database Management, SSDBM 2017
    CountryUnited States
    CityChicago
    Period6/27/176/29/17

    Keywords

    • Accountability
    • Data
    • Data ethics
    • Data science for social good
    • Fairness
    • Responsibly
    • Transparency

    ASJC Scopus subject areas

    • Software
    • Human-Computer Interaction
    • Computer Vision and Pattern Recognition
    • Computer Networks and Communications

    Fingerprint Dive into the research topics of 'Measuring fairness in ranked outputs'. Together they form a unique fingerprint.

  • Cite this

    Yang, K., & Stoyanovich, J. (2017). Measuring fairness in ranked outputs. In SSDBM 2017: 29th International Conference on Scientific and Statistical Database Management [a22] (ACM International Conference Proceeding Series; Vol. Part F128636). Association for Computing Machinery. https://doi.org/10.1145/3085504.3085526