TY - GEN
T1 - Improved learning of Gaussian-Bernoulli restricted Boltzmann machines
AU - Cho, Kyung Hyun
AU - Ilin, Alexander
AU - Raiko, Tapani
PY - 2011
Y1 - 2011
N2 - We propose a few remedies to improve training of Gaussian-Bernoulli restricted Boltzmann machines (GBRBM), which is known to be difficult. Firstly, we use a different parameterization of the energy function, which allows for more intuitive interpretation of the parameters and facilitates learning. Secondly, we propose parallel tempering learning for GBRBM. Lastly, we use an adaptive learning rate which is selected automatically in order to stabilize training. Our extensive experiments show that the proposed improvements indeed remove most of the difficulties encountered when training GBRBMs using conventional methods.
AB - We propose a few remedies to improve training of Gaussian-Bernoulli restricted Boltzmann machines (GBRBM), which is known to be difficult. Firstly, we use a different parameterization of the energy function, which allows for more intuitive interpretation of the parameters and facilitates learning. Secondly, we propose parallel tempering learning for GBRBM. Lastly, we use an adaptive learning rate which is selected automatically in order to stabilize training. Our extensive experiments show that the proposed improvements indeed remove most of the difficulties encountered when training GBRBMs using conventional methods.
KW - Adaptive Learning Rate
KW - Gaussian-Bernoulli Restricted Boltzmann Machine
KW - Parallel Tempering
KW - Restricted Boltzmann Machine
UR - http://www.scopus.com/inward/record.url?scp=79959342724&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=79959342724&partnerID=8YFLogxK
U2 - 10.1007/978-3-642-21735-7_2
DO - 10.1007/978-3-642-21735-7_2
M3 - Conference contribution
AN - SCOPUS:79959342724
SN - 9783642217340
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 10
EP - 17
BT - Artificial Neural Networks and Machine Learning, ICANN 2011 - 21st International Conference on Artificial Neural Networks, Proceedings
T2 - 21st International Conference on Artificial Neural Networks, ICANN 2011
Y2 - 14 June 2011 through 17 June 2011
ER -