Simultaneous learning of trees and representations for extreme classification and density estimation

Yacine Jernite, Anna Choromanska, David Sontag

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We consider multi-class classification where the predictor has a hierarchical structure that allows for a very large number of labels both at train and test time. The predictive power of such models can heavily depend on the structure of the tree, and although past work showed how to learn the tree structure, it expected that the feature vectors remained static. We provide a novel algorithm to simultaneously perform representation learning for the input data and learning of the hierarchical predictor. Our approach optimizes an objective function which favors balanced and easily-separable multi-way node partitions. We theoretically analyze this objective, showing that it gives rise to a boosting style property and a bound on classification error. We next show how to extend the algorithm to conditional density estimation. We empirically validate both variants of the algorithm on text classification and language modeling, respectively, and show that they compare favorably to common baselines in terms of accuracy and running time.

Original languageEnglish (US)
Title of host publication34th International Conference on Machine Learning, ICML 2017
PublisherInternational Machine Learning Society (IMLS)
Pages2613-2633
Number of pages21
ISBN (Electronic)9781510855144
StatePublished - 2017
Event34th International Conference on Machine Learning, ICML 2017 - Sydney, Australia
Duration: Aug 6 2017Aug 11 2017

Publication series

Name34th International Conference on Machine Learning, ICML 2017
Volume4

Other

Other34th International Conference on Machine Learning, ICML 2017
CountryAustralia
CitySydney
Period8/6/178/11/17

ASJC Scopus subject areas

  • Computational Theory and Mathematics
  • Human-Computer Interaction
  • Software

Fingerprint Dive into the research topics of 'Simultaneous learning of trees and representations for extreme classification and density estimation'. Together they form a unique fingerprint.

  • Cite this

    Jernite, Y., Choromanska, A., & Sontag, D. (2017). Simultaneous learning of trees and representations for extreme classification and density estimation. In 34th International Conference on Machine Learning, ICML 2017 (pp. 2613-2633). (34th International Conference on Machine Learning, ICML 2017; Vol. 4). International Machine Learning Society (IMLS).