Unsupervised learning of hierarchical representations with convolutional deep belief networks

Honglak Lee, Roger Grosse, Rajesh Ranganath, Andrew Y. Ng

Research output: Contribution to journalArticlepeer-review

Abstract

There has been much interest in unsupervised learning of hierarchical generative models such as deep belief networks (DBNs); however, scaling such models to full-sized, highdimensional images remains a difficult problem. To address this problem, we present the convolutional deep belief network, a hierarchical generative model that scales to realistic image sizes. This model is translation-invariant and supports efficient bottom-up and top-down probabilistic inference. Key to our approach is probabilistic max-pooling, a novel technique that shrinks the representations of higher layers in a probabilistically sound way. Our experiments show that the algorithm learns useful high-level visual features, such as object parts, from unlabeled images of objects and natural scenes. We demonstrate excellent performance on several visual recognition tasks and show that our model can perform hierarchical (bottom-up and top-down) inference over full-sized images.

Original languageEnglish (US)
Pages (from-to)95-103
Number of pages9
JournalCommunications of the ACM
Volume54
Issue number10
DOIs
StatePublished - Oct 2011

ASJC Scopus subject areas

  • Computer Science(all)

Fingerprint Dive into the research topics of 'Unsupervised learning of hierarchical representations with convolutional deep belief networks'. Together they form a unique fingerprint.

Cite this