Surrogate decision tree visualization interpreting and visualizing black-box classification models with surrogate decision tree

With the growing interest towards the application of Machine Learning techniques to many application domains, the need for transparent and interpretable ML is getting stronger. Visualizations methods can help model developers understand and refine ML models by making the logic of a given model visible and interactive. In this paper we describe a visual analytics tool we developed to support developers and domain experts (with little to no expertise in ML) in understanding the logic of a ML model without having access to the internal structure of the model (i.e., a model-agnostic method). The method is based on the creation of a “surrogate” decision tree which simulates the behavior of the black-box model of interest and presents readable rules to the end-users. We evaluate the effectiveness of the method with a preliminary user study and analysis of the level of fidelity the surrogate decision tree can reach with respect to the original model.