Measuring fairness in ranked outputs

Ke Yang, Julia Stoyanovich

    Research output: Chapter in Book/Report/Conference proceedingConference contribution


    Ranking and scoring are ubiquitous. We consider the setting in which an institution, called a ranker, evaluates a set of individuals based on demographic, behavioral or other characteristics. The final output is a ranking that represents the relative quality of the individuals. While automatic and therefore seemingly objective, rankers can, and often do, discriminate against individuals and systematically disadvantage members of protected groups. This warrants a careful study of the fairness of a ranking scheme, to enable data science for social good applications, among others. In this paper we propose fairness measures for ranked outputs. We develop a data generation procedure that allows us to systematically control the degree of unfairness in the output, and study the behavior of our measures on these datasets. We then apply our proposed measures to several real datasets, and detect cases of bias. Finally, we show preliminary results of incorporating our ranked fairness measures into an optimization framework, and show potential for improving fairness of ranked outputs while maintaining accuracy.

    Original languageEnglish (US)
    Title of host publicationSSDBM 2017
    Subtitle of host publication29th International Conference on Scientific and Statistical Database Management
    PublisherAssociation for Computing Machinery
    ISBN (Electronic)9781450352826
    StatePublished - Jun 27 2017
    Event29th International Conference on Scientific and Statistical Database Management, SSDBM 2017 - Chicago, United States
    Duration: Jun 27 2017Jun 29 2017

    Publication series

    NameACM International Conference Proceeding Series
    VolumePart F128636


    Other29th International Conference on Scientific and Statistical Database Management, SSDBM 2017
    Country/TerritoryUnited States


    • Accountability
    • Data
    • Data ethics
    • Data science for social good
    • Fairness
    • Responsibly
    • Transparency

    ASJC Scopus subject areas

    • Software
    • Human-Computer Interaction
    • Computer Vision and Pattern Recognition
    • Computer Networks and Communications


    Dive into the research topics of 'Measuring fairness in ranked outputs'. Together they form a unique fingerprint.

    Cite this