Interpreting black-box classifiers using instance-level visual explanations

Paolo Tamagnini; Josua Krause; Aritra Dasgupta; Enrico Bertini

doi:10.1145/3077257.3077260

Interpreting black-box classifiers using instance-level visual explanations

Paolo Tamagnini, Josua Krause, Aritra Dasgupta, Enrico Bertini

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

To realize the full potential of machine learning in diverse real-world domains, it is necessary for model predictions to be readily interpretable and actionable for the human in the loop. Analysts, who are the users but not the developers of machine learning models, often do not trust a model because of the lack of transparency in associating predictions with the underlying data space. To address this problem, we propose Rivelo, a visual analytics interface that enables analysts to understand the causes behind predictions of binary classifiers by interactively exploring a set of instance-level explanations. These explanations are model-agnostic, treating a model as a black box, and they help analysts in interactively probing the high-dimensional binary data space for detecting features relevant to predictions. We demonstrate the utility of the interface with a case study analyzing a random forest model on the sentiment of Yelp reviews about doctors.

Original language	English (US)
Title of host publication	Proceedings of the 2nd Workshop on Human-In-the-Loop Data Analytics, HILDA 2017
Publisher	Association for Computing Machinery, Inc
ISBN (Electronic)	9781450350297
DOIs	https://doi.org/10.1145/3077257.3077260
State	Published - May 14 2017
Event	2nd Workshop on Human-In-the-Loop Data Analytics, HILDA 2017 - Chicago, United States Duration: May 14 2017 → …

Publication series

Name	Proceedings of the 2nd Workshop on Human-In-the-Loop Data Analytics, HILDA 2017

Other

Other	2nd Workshop on Human-In-the-Loop Data Analytics, HILDA 2017
Country/Territory	United States
City	Chicago
Period	5/14/17 → …

Keywords

Classification
Explanation
Machine learning
Visual analytics

ASJC Scopus subject areas

Computational Theory and Mathematics
Computer Science Applications
Information Systems

Access to Document

10.1145/3077257.3077260

Cite this

Tamagnini, P., Krause, J., Dasgupta, A., & Bertini, E. (2017). Interpreting black-box classifiers using instance-level visual explanations. In Proceedings of the 2nd Workshop on Human-In-the-Loop Data Analytics, HILDA 2017 Article 3077260 (Proceedings of the 2nd Workshop on Human-In-the-Loop Data Analytics, HILDA 2017). Association for Computing Machinery, Inc. https://doi.org/10.1145/3077257.3077260

Interpreting black-box classifiers using instance-level visual explanations. / Tamagnini, Paolo; Krause, Josua; Dasgupta, Aritra et al.
Proceedings of the 2nd Workshop on Human-In-the-Loop Data Analytics, HILDA 2017. Association for Computing Machinery, Inc, 2017. 3077260 (Proceedings of the 2nd Workshop on Human-In-the-Loop Data Analytics, HILDA 2017).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Tamagnini, P, Krause, J, Dasgupta, A & Bertini, E 2017, Interpreting black-box classifiers using instance-level visual explanations. in Proceedings of the 2nd Workshop on Human-In-the-Loop Data Analytics, HILDA 2017., 3077260, Proceedings of the 2nd Workshop on Human-In-the-Loop Data Analytics, HILDA 2017, Association for Computing Machinery, Inc, 2nd Workshop on Human-In-the-Loop Data Analytics, HILDA 2017, Chicago, United States, 5/14/17. https://doi.org/10.1145/3077257.3077260

Tamagnini P, Krause J, Dasgupta A, Bertini E. Interpreting black-box classifiers using instance-level visual explanations. In Proceedings of the 2nd Workshop on Human-In-the-Loop Data Analytics, HILDA 2017. Association for Computing Machinery, Inc. 2017. 3077260. (Proceedings of the 2nd Workshop on Human-In-the-Loop Data Analytics, HILDA 2017). doi: 10.1145/3077257.3077260

@inproceedings{10c86a7d941646a7be87e13a99e5c774,

title = "Interpreting black-box classifiers using instance-level visual explanations",

abstract = "To realize the full potential of machine learning in diverse real-world domains, it is necessary for model predictions to be readily interpretable and actionable for the human in the loop. Analysts, who are the users but not the developers of machine learning models, often do not trust a model because of the lack of transparency in associating predictions with the underlying data space. To address this problem, we propose Rivelo, a visual analytics interface that enables analysts to understand the causes behind predictions of binary classifiers by interactively exploring a set of instance-level explanations. These explanations are model-agnostic, treating a model as a black box, and they help analysts in interactively probing the high-dimensional binary data space for detecting features relevant to predictions. We demonstrate the utility of the interface with a case study analyzing a random forest model on the sentiment of Yelp reviews about doctors.",

keywords = "Classification, Explanation, Machine learning, Visual analytics",

author = "Paolo Tamagnini and Josua Krause and Aritra Dasgupta and Enrico Bertini",

note = "Publisher Copyright: {\textcopyright} 2017 ACM.; 2nd Workshop on Human-In-the-Loop Data Analytics, HILDA 2017 ; Conference date: 14-05-2017",

year = "2017",

month = may,

day = "14",

doi = "10.1145/3077257.3077260",

language = "English (US)",

series = "Proceedings of the 2nd Workshop on Human-In-the-Loop Data Analytics, HILDA 2017",

publisher = "Association for Computing Machinery, Inc",

booktitle = "Proceedings of the 2nd Workshop on Human-In-the-Loop Data Analytics, HILDA 2017",

}

TY - GEN

T1 - Interpreting black-box classifiers using instance-level visual explanations

AU - Tamagnini, Paolo

AU - Krause, Josua

AU - Dasgupta, Aritra

AU - Bertini, Enrico

PY - 2017/5/14

Y1 - 2017/5/14

N2 - To realize the full potential of machine learning in diverse real-world domains, it is necessary for model predictions to be readily interpretable and actionable for the human in the loop. Analysts, who are the users but not the developers of machine learning models, often do not trust a model because of the lack of transparency in associating predictions with the underlying data space. To address this problem, we propose Rivelo, a visual analytics interface that enables analysts to understand the causes behind predictions of binary classifiers by interactively exploring a set of instance-level explanations. These explanations are model-agnostic, treating a model as a black box, and they help analysts in interactively probing the high-dimensional binary data space for detecting features relevant to predictions. We demonstrate the utility of the interface with a case study analyzing a random forest model on the sentiment of Yelp reviews about doctors.

AB - To realize the full potential of machine learning in diverse real-world domains, it is necessary for model predictions to be readily interpretable and actionable for the human in the loop. Analysts, who are the users but not the developers of machine learning models, often do not trust a model because of the lack of transparency in associating predictions with the underlying data space. To address this problem, we propose Rivelo, a visual analytics interface that enables analysts to understand the causes behind predictions of binary classifiers by interactively exploring a set of instance-level explanations. These explanations are model-agnostic, treating a model as a black box, and they help analysts in interactively probing the high-dimensional binary data space for detecting features relevant to predictions. We demonstrate the utility of the interface with a case study analyzing a random forest model on the sentiment of Yelp reviews about doctors.

KW - Classification

KW - Explanation

KW - Machine learning

KW - Visual analytics

UR - http://www.scopus.com/inward/record.url?scp=85021682673&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85021682673&partnerID=8YFLogxK

U2 - 10.1145/3077257.3077260

DO - 10.1145/3077257.3077260

M3 - Conference contribution

AN - SCOPUS:85021682673

T3 - Proceedings of the 2nd Workshop on Human-In-the-Loop Data Analytics, HILDA 2017

BT - Proceedings of the 2nd Workshop on Human-In-the-Loop Data Analytics, HILDA 2017

PB - Association for Computing Machinery, Inc

T2 - 2nd Workshop on Human-In-the-Loop Data Analytics, HILDA 2017

Y2 - 14 May 2017

ER -

Interpreting black-box classifiers using instance-level visual explanations

Abstract

Publication series

Other

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this