TY - GEN
T1 - A dual perturbation approach for differential private admm-based distributed empirical risk minimization
AU - Zhang, Tao
AU - Zhu, Quanyan
N1 - Funding Information:
We would like to thank National Science Foundation (CNS-1544782) and NYU University Research Challenge Fund (URCF) that partially support this research.
Publisher Copyright:
© 2016 ACM.
PY - 2016/10/28
Y1 - 2016/10/28
N2 - The rapid growth of data has raised the importance of privacy-preserving techniques in distributed machine learning. In this paper, we develop a privacy-preserving method to a class of regularized empirical risk minimization (ERM) machine learning problems. We first decentralize the learning algorithm using the alternating direction method of multipliers (ADMM), and propose the method of dual variable perturbation to provide dynamic differential privacy. The mechanism leads to a privacy-preserving algorithm under mild conditions of the convexity and differentiability of the loss function and the regularizer. We study the performance of the algorithm measured by the number of data points required to achieve a bounded error. To design an optimal privacy mechanism, we analyze the fundamental tradeoff between privacy and accuracy, and provide guidelines to choose privacy parameters. Numerical experiments using the realworld database are performed to corroborate the results on the privacy and utility tradeoffs and design.
AB - The rapid growth of data has raised the importance of privacy-preserving techniques in distributed machine learning. In this paper, we develop a privacy-preserving method to a class of regularized empirical risk minimization (ERM) machine learning problems. We first decentralize the learning algorithm using the alternating direction method of multipliers (ADMM), and propose the method of dual variable perturbation to provide dynamic differential privacy. The mechanism leads to a privacy-preserving algorithm under mild conditions of the convexity and differentiability of the loss function and the regularizer. We study the performance of the algorithm measured by the number of data points required to achieve a bounded error. To design an optimal privacy mechanism, we analyze the fundamental tradeoff between privacy and accuracy, and provide guidelines to choose privacy parameters. Numerical experiments using the realworld database are performed to corroborate the results on the privacy and utility tradeoffs and design.
KW - ADMM
KW - Differential privacy
KW - Distributed optimization
KW - Machine learning
KW - Privacy tradeoffs
UR - http://www.scopus.com/inward/record.url?scp=85002424699&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85002424699&partnerID=8YFLogxK
U2 - 10.1145/2996758.2996762
DO - 10.1145/2996758.2996762
M3 - Conference contribution
AN - SCOPUS:85002424699
T3 - AISec 2016 - Proceedings of the 2016 ACM Workshop on Artificial Intelligence and Security, co-located with CCS 2016
SP - 129
EP - 137
BT - AISec 2016 - Proceedings of the 2016 ACM Workshop on Artificial Intelligence and Security, co-located with CCS 2016
PB - Association for Computing Machinery, Inc
T2 - 9th ACM Workshop on Artificial Intelligence and Security, AISec 2016
Y2 - 28 October 2016
ER -