Tikhonov-type regularization for restricted Boltzmann machines

Kyung Hyun Cho, Alexander Ilin, Tapani Raiko

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In this paper, we study a Tikhonov-type regularization for restricted Boltzmann machines (RBM). We present two alternative formulations of the Tikhonov-type regularization which encourage an RBM to learn a smoother probability distribution. Both formulations turn out to be combinations of the widely used weight-decay and sparsity regularization. We empirically evaluate the effect of the proposed regularization schemes and show that the use of them could help extracting better discriminative features with sparser hidden activation probabilities.

Original languageEnglish (US)
Title of host publicationArtificial Neural Networks and Machine Learning, ICANN 2012 - 22nd International Conference on Artificial Neural Networks, Proceedings
Pages81-88
Number of pages8
EditionPART 1
DOIs
StatePublished - 2012
Event22nd International Conference on Artificial Neural Networks, ICANN 2012 - Lausanne, Switzerland
Duration: Sep 11 2012Sep 14 2012

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
NumberPART 1
Volume7552 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other22nd International Conference on Artificial Neural Networks, ICANN 2012
Country/TerritorySwitzerland
CityLausanne
Period9/11/129/14/12

Keywords

  • Restricted Boltzmann Machine
  • Tikhonov Regularization

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint

Dive into the research topics of 'Tikhonov-type regularization for restricted Boltzmann machines'. Together they form a unique fingerprint.

Cite this