PipelineProfiler: A Visual Analytics Tool for the Exploration of AutoML Pipelines

Jorge Piazentin Ono, Sonia Castelo, Roque Lopez, Enrico Bertini, Juliana Freire, Claudio Silva

Research output: Contribution to journalArticlepeer-review

Abstract

In recent years, a wide variety of automated machine learning (AutoML) methods have been proposed to generate end-to-end ML pipelines. While these techniques facilitate the creation of models, given their black-box nature, the complexity of the underlying algorithms, and the large number of pipelines they derive, they are difficult for developers to debug. It is also challenging for machine learning experts to select an AutoML system that is well suited for a given problem. In this paper, we present the Pipeline Profiler, an interactive visualization tool that allows the exploration and comparison of the solution space of machine learning (ML) pipelines produced by AutoML systems. PipelineProfiler is integrated with Jupyter Notebook and can be combined with common data science tools to enable a rich set of analyses of the ML pipelines, providing users a better understanding of the algorithms that generated them as well as insights into how they can be improved. We demonstrate the utility of our tool through use cases where PipelineProfiler is used to better understand and improve a real-world AutoML system. Furthermore, we validate our approach by presenting a detailed analysis of a think-aloud experiment with six data scientists who develop and evaluate AutoML tools.

Original languageEnglish (US)
Article number9222086
Pages (from-to)390-400
Number of pages11
JournalIEEE Transactions on Visualization and Computer Graphics
Volume27
Issue number2
DOIs
StatePublished - Feb 2021

Keywords

  • Automatic Machine Learning
  • Model Evaluation
  • Pipeline Visualization

ASJC Scopus subject areas

  • Software
  • Signal Processing
  • Computer Vision and Pattern Recognition
  • Computer Graphics and Computer-Aided Design

Fingerprint Dive into the research topics of 'PipelineProfiler: A Visual Analytics Tool for the Exploration of AutoML Pipelines'. Together they form a unique fingerprint.

Cite this