HERB+: Evolving an Industrial-Strength Privacy-Preserving Machine Learning Framework

Supervised machine learning does not hold without data. However, the needed data can be distributed in different locations and are non-shareable under privacy constraints. Methods to circumvent disclosure restrictions in collaborative machine learning are in strong demand. Thus, we propose HERB+ (Homomorphic Encryption for Random forest and gradient Boosting plus), a confidential learning framework for tree-based models under the scenario of vertically dispersed data. While previous related work focused on a specific algorithm, this work presents a wide variety of privacy-preserved and distributed tree-based algorithms (i.e., Decision Tree, Random Forest, and Gradient Boosting Decision Trees for both classification and regression tasks). HERB+ provides the most detailed and general discussions on using Fully Homomorphic Encryption for computing distributed tree-based algorithms during the training process. Our experiments show that although the learning protocols' efficiencies are not optimal, the predictive performance and privacy are preserved. The results imply that practitioners can overcome the barrier of data sharing and produce tree-based models for data-heavy domains with strict privacy requirements, such as Health Prediction, Fraud Detection, and Risk Evaluation.