Improved learning of Gaussian-Bernoulli restricted Boltzmann machines

Kyung Hyun Cho, Alexander Ilin, Tapani Raiko

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We propose a few remedies to improve training of Gaussian-Bernoulli restricted Boltzmann machines (GBRBM), which is known to be difficult. Firstly, we use a different parameterization of the energy function, which allows for more intuitive interpretation of the parameters and facilitates learning. Secondly, we propose parallel tempering learning for GBRBM. Lastly, we use an adaptive learning rate which is selected automatically in order to stabilize training. Our extensive experiments show that the proposed improvements indeed remove most of the difficulties encountered when training GBRBMs using conventional methods.

Original languageEnglish (US)
Title of host publicationArtificial Neural Networks and Machine Learning, ICANN 2011 - 21st International Conference on Artificial Neural Networks, Proceedings
Pages10-17
Number of pages8
EditionPART 1
DOIs
StatePublished - 2011
Event21st International Conference on Artificial Neural Networks, ICANN 2011 - Espoo, Finland
Duration: Jun 14 2011Jun 17 2011

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
NumberPART 1
Volume6791 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other21st International Conference on Artificial Neural Networks, ICANN 2011
CountryFinland
CityEspoo
Period6/14/116/17/11

Keywords

  • Adaptive Learning Rate
  • Gaussian-Bernoulli Restricted Boltzmann Machine
  • Parallel Tempering
  • Restricted Boltzmann Machine

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint Dive into the research topics of 'Improved learning of Gaussian-Bernoulli restricted Boltzmann machines'. Together they form a unique fingerprint.

Cite this