Exposing the probabilistic causal structure of discrimination

Francesco Bonchi, Sara Hajian, Bud Mishra, Daniele Ramazzotti

Research output: Contribution to journalArticlepeer-review


Discrimination discovery from data is an important data mining task, whose goal is to identify patterns of illegal and unethical discriminatory activities against protected-by-law groups, e.g., ethnic minorities. While any legally valid proof of discrimination requires evidence of causality, the state-of-the-art methods are essentially correlation based, albeit, as it is well known, correlation does not imply causation. In this paper, we take a principled causal approach to discrimination detection following Suppes’ probabilistic causation theory. In particular, we define a method to extract, from a dataset of historical decision records, the causal structures existing among the attributes in the data. The result is a type of constrained Bayesian network, which we dub Suppes-Bayes causal network (SBCN). Next, we develop a toolkit of methods based on random walks on top of the SBCN, addressing different anti-discrimination legal concepts, such as direct and indirect discrimination, group and individual discrimination, genuine requirement, and favoritism. Our experiments on real-world datasets confirm the inferential power of our approach in all these different tasks.

Original languageEnglish (US)
Pages (from-to)1-21
Number of pages21
JournalInternational Journal of Data Science and Analytics
Issue number1
StatePublished - Feb 1 2017


  • Algorithmic discrimination
  • Discrimination discovery
  • Random walks
  • constrained Bayesian network
  • probabilistic causation

ASJC Scopus subject areas

  • Information Systems
  • Modeling and Simulation
  • Computer Science Applications
  • Computational Theory and Mathematics
  • Applied Mathematics


Dive into the research topics of 'Exposing the probabilistic causal structure of discrimination'. Together they form a unique fingerprint.

Cite this