Locally convex kernel mixtures: Bayesian subspace learning

Duy Hoang Thai, Hau Tieng Wu, David B. Dunson

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Kernel mixture models are routinely used for density estimation. However, in multivariate settings, issues arise in efficiently approximating lower-dimensional structure in the data. For example, it is common to suppose that the density is concentrated near a lower-dimensional non-linear subspace or manifold. Typical kernels used to locally approximate such subspaces are inflexible, so that a large number of components are often needed. We propose a novel class of LOcally COnvex (LOCO) kernels that are flexible in adapting to nonlinear local structure. LOCO kernels are induced by introducing random knots within local neighborhoods, and generating data as a random convex combination of these knots with adaptive weights and an additive noise. For identifiability, we constrain all observations from a particular component to have the same mean. For Bayesian inference subject to this constraint, we develop a hybrid Gibbs sampler and optimization algorithm that incorporates a Lagrange multiplier within a splitting method. The resulting LOCO algorithm is shown to dramatically outperform typical Gaussian mixture models in challenging examples.

Original languageEnglish (US)
Title of host publicationProceedings - 18th IEEE International Conference on Machine Learning and Applications, ICMLA 2019
EditorsM. Arif Wani, Taghi M. Khoshgoftaar, Dingding Wang, Huanjing Wang, Naeem Seliya
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages272-275
Number of pages4
ISBN (Electronic)9781728145495
DOIs
StatePublished - Dec 2019
Event18th IEEE International Conference on Machine Learning and Applications, ICMLA 2019 - Boca Raton, United States
Duration: Dec 16 2019Dec 19 2019

Publication series

NameProceedings - 18th IEEE International Conference on Machine Learning and Applications, ICMLA 2019

Conference

Conference18th IEEE International Conference on Machine Learning and Applications, ICMLA 2019
Country/TerritoryUnited States
CityBoca Raton
Period12/16/1912/19/19

Keywords

  • Bayesian Classification
  • Density Estimation
  • Kernel Method
  • Majorization-Maximization
  • Mixture Models

ASJC Scopus subject areas

  • Strategy and Management
  • Artificial Intelligence
  • Computer Science Applications
  • Decision Sciences (miscellaneous)
  • Signal Processing
  • Media Technology

Fingerprint

Dive into the research topics of 'Locally convex kernel mixtures: Bayesian subspace learning'. Together they form a unique fingerprint.

Cite this