Gaussian-Bernoulli deep Boltzmann machine

Kyung Hyun Cho, Tapani Raiko, Alexander Ilin

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In this paper, we study a model that we call Gaussian-Bernoulli deep Boltzmann machine (GDBM) and discuss potential improvements in training the model. GDBM is designed to be applicable to continuous data and it is constructed from Gaussian-Bernoulli restricted Boltzmann machine (GRBM) by adding multiple layers of binary hidden neurons. The studied improvements of the learning algorithm for GDBM include parallel tempering, enhanced gradient, adaptive learning rate and layer-wise pretraining. We empirically show that they help avoid some of the common difficulties found in training deep Boltzmann machines such as divergence of learning, the difficulty in choosing right learning rate scheduling, and the existence of meaningless higher layers.

Original languageEnglish (US)
Title of host publication2013 International Joint Conference on Neural Networks, IJCNN 2013
DOIs
StatePublished - Dec 1 2013
Event2013 International Joint Conference on Neural Networks, IJCNN 2013 - Dallas, TX, United States
Duration: Aug 4 2013Aug 9 2013

Publication series

NameProceedings of the International Joint Conference on Neural Networks

Other

Other2013 International Joint Conference on Neural Networks, IJCNN 2013
CountryUnited States
CityDallas, TX
Period8/4/138/9/13

ASJC Scopus subject areas

  • Software
  • Artificial Intelligence

Fingerprint Dive into the research topics of 'Gaussian-Bernoulli deep Boltzmann machine'. Together they form a unique fingerprint.

  • Cite this

    Cho, K. H., Raiko, T., & Ilin, A. (2013). Gaussian-Bernoulli deep Boltzmann machine. In 2013 International Joint Conference on Neural Networks, IJCNN 2013 [6706831] (Proceedings of the International Joint Conference on Neural Networks). https://doi.org/10.1109/IJCNN.2013.6706831