TY - GEN
T1 - Efficient large-scale distributed training of conditional maximum entropy models
AU - Mann, Gideon
AU - McDonald, Ryan
AU - Mohri, Mehryar
AU - Silberman, Nathan
AU - Walker, Daniel D.
PY - 2009
Y1 - 2009
N2 - Training conditional maximum entropy models on massive data sets requires significant computational resources. We examine three common distributed training methods for conditional maxent: a distributed gradient computation method, a majority vote method, and a mixture weight method. We analyze and compare the CPU and network time complexity of each of these methods and present a theoretical analysis of conditional maxent models, including a study of the convergence of the mixture weight method, the most resource-efficient technique. We also report the results of large-scale experiments comparing these three methods which demonstrate the benefits of the mixture weight method: this method consumes less resources, while achieving a performance comparable to that of standard approaches.
AB - Training conditional maximum entropy models on massive data sets requires significant computational resources. We examine three common distributed training methods for conditional maxent: a distributed gradient computation method, a majority vote method, and a mixture weight method. We analyze and compare the CPU and network time complexity of each of these methods and present a theoretical analysis of conditional maxent models, including a study of the convergence of the mixture weight method, the most resource-efficient technique. We also report the results of large-scale experiments comparing these three methods which demonstrate the benefits of the mixture weight method: this method consumes less resources, while achieving a performance comparable to that of standard approaches.
UR - http://www.scopus.com/inward/record.url?scp=80052652249&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=80052652249&partnerID=8YFLogxK
M3 - Conference contribution
AN - SCOPUS:80052652249
SN - 9781615679119
T3 - Advances in Neural Information Processing Systems 22 - Proceedings of the 2009 Conference
SP - 1231
EP - 1239
BT - Advances in Neural Information Processing Systems 22 - Proceedings of the 2009 Conference
PB - Neural Information Processing Systems
T2 - 23rd Annual Conference on Neural Information Processing Systems, NIPS 2009
Y2 - 7 December 2009 through 10 December 2009
ER -