Abstract
We introduce a view of unsupervised learning that integrates probabilistic and non-probabilistic methods for clustering, dimensionality reduction, and feature extraction in a unified framework. In this framework, an energy function associates low energies to input points that are similar to training samples, and high energies to unobserved points. Learning consists in minimizing the energies of training samples while ensuring that the energies of unobserved ones are higher. Some traditional methods construct the architecture so that only a small number of points can have low energy, while other methods explicitly "pull up" on the energies of unobserved points. In probabilistic methods the energy of unobserved points is pulled by minimizing the log partition function, an expensive, and sometimes intractable process. We explore different and more efficient methods using an energy-based approach. In particular, we show that a simple solution is to restrict the amount of information contained in codes that represent the data. We demonstrate such a method by training it on natural image patches and by applying to image denoising.
Original language | English (US) |
---|---|
Pages (from-to) | 371-379 |
Number of pages | 9 |
Journal | Journal of Machine Learning Research |
Volume | 2 |
State | Published - 2007 |
Event | 11th International Conference on Artificial Intelligence and Statistics, AISTATS 2007 - San Juan, Puerto Rico Duration: Mar 21 2007 → Mar 24 2007 |
ASJC Scopus subject areas
- Software
- Control and Systems Engineering
- Statistics and Probability
- Artificial Intelligence