Scene parsing with multiscale feature learning, purity trees, and optimal covers

Clément Farabet, Camille Couprie, Laurent Najman, Yann LeCun

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Scene parsing consists in labeling each pixel in an image with the category of the object it belongs to. We propose a method that uses a multiscale convolutional network trained from raw pixels to extract dense feature vectors that encode regions of multiple sizes centered on each pixel. The method alleviates the need for engineered features. In parallel to feature extraction, a tree of segments is computed from a graph of pixel dissimilarities. The feature vectors associated with the segments covered by each node in the tree are aggregated and fed to a classifier which produces an estimate of the distribution of object categories contained in the segment. A subset of tree nodes that cover the image are then selected so as to maximize the average "purity" of the class distributions, hence maximizing the overall likelihood that each segment will contain a single object. The system yields record accuracies on the the Sift Flow Dataset (33 classes) and the Barcelona Dataset (170 classes) and near-record accuracy on the Stanford Background Dataset (8 classes), while being an order of magnitude faster than competing approaches, producing a 320 x 240 image labeling in less than 1 second, including feature extraction.

Original languageEnglish (US)
Title of host publicationProceedings of the 29th International Conference on Machine Learning, ICML 2012
Pages575-582
Number of pages8
StatePublished - 2012
Event29th International Conference on Machine Learning, ICML 2012 - Edinburgh, United Kingdom
Duration: Jun 26 2012Jul 1 2012

Publication series

NameProceedings of the 29th International Conference on Machine Learning, ICML 2012
Volume1

Other

Other29th International Conference on Machine Learning, ICML 2012
CountryUnited Kingdom
CityEdinburgh
Period6/26/127/1/12

ASJC Scopus subject areas

  • Human-Computer Interaction
  • Education

Fingerprint Dive into the research topics of 'Scene parsing with multiscale feature learning, purity trees, and optimal covers'. Together they form a unique fingerprint.

  • Cite this

    Farabet, C., Couprie, C., Najman, L., & LeCun, Y. (2012). Scene parsing with multiscale feature learning, purity trees, and optimal covers. In Proceedings of the 29th International Conference on Machine Learning, ICML 2012 (pp. 575-582). (Proceedings of the 29th International Conference on Machine Learning, ICML 2012; Vol. 1).