Learning a predictable and generative vector representation for objects

Rohit Girdhar; David F. Fouhey; Mikel Rodriguez; Abhinav Gupta

doi:10.1007/978-3-319-46466-4_29

Learning a predictable and generative vector representation for objects

Rohit Girdhar, David F. Fouhey, Mikel Rodriguez, Abhinav Gupta

Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

What is a good vector representation of an object? We believe that it should be generative in 3D, in the sense that it can produce new 3D objects; as well as be predictable from 2D, in the sense that it can be perceived from 2D images. We propose a novel architecture, called the TL-embedding network, to learn an embedding space with these properties. The network consists of two components: (a) an autoencoder that ensures the representation is generative; and (b) a convolutional network that ensures the representation is predictable. This enables tackling a number of tasks including voxel prediction from 2D images and 3D model retrieval. Extensive experimental analysis demonstrates the usefulness and versatility of this embedding.

Original language	English (US)
Title of host publication	Computer Vision - 14th European Conference, ECCV 2016, Proceedings
Editors	Bastian Leibe, Jiri Matas, Nicu Sebe, Max Welling
Publisher	Springer Verlag
Pages	484-499
Number of pages	16
ISBN (Print)	9783319464657
DOIs	https://doi.org/10.1007/978-3-319-46466-4_29
State	Published - 2016
Event	14th European Conference on Computer Vision, ECCV 2016 - Amsterdam, Netherlands Duration: Oct 8 2016 → Oct 16 2016

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	9910 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	14th European Conference on Computer Vision, ECCV 2016
Country/Territory	Netherlands
City	Amsterdam
Period	10/8/16 → 10/16/16

ASJC Scopus subject areas

Theoretical Computer Science
Computer Science(all)

Access to Document

10.1007/978-3-319-46466-4_29

Cite this

Girdhar, R., Fouhey, D. F., Rodriguez, M., & Gupta, A. (2016). Learning a predictable and generative vector representation for objects. In B. Leibe, J. Matas, N. Sebe, & M. Welling (Eds.), Computer Vision - 14th European Conference, ECCV 2016, Proceedings (pp. 484-499). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9910 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-319-46466-4_29

Learning a predictable and generative vector representation for objects. / Girdhar, Rohit; Fouhey, David F.; Rodriguez, Mikel et al.
Computer Vision - 14th European Conference, ECCV 2016, Proceedings. ed. / Bastian Leibe; Jiri Matas; Nicu Sebe; Max Welling. Springer Verlag, 2016. p. 484-499 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9910 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Girdhar, R, Fouhey, DF, Rodriguez, M & Gupta, A 2016, Learning a predictable and generative vector representation for objects. in B Leibe, J Matas, N Sebe & M Welling (eds), Computer Vision - 14th European Conference, ECCV 2016, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 9910 LNCS, Springer Verlag, pp. 484-499, 14th European Conference on Computer Vision, ECCV 2016, Amsterdam, Netherlands, 10/8/16. https://doi.org/10.1007/978-3-319-46466-4_29

Girdhar R, Fouhey DF, Rodriguez M, Gupta A. Learning a predictable and generative vector representation for objects. In Leibe B, Matas J, Sebe N, Welling M, editors, Computer Vision - 14th European Conference, ECCV 2016, Proceedings. Springer Verlag. 2016. p. 484-499. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-319-46466-4_29

Girdhar, Rohit ; Fouhey, David F. ; Rodriguez, Mikel et al. / Learning a predictable and generative vector representation for objects. Computer Vision - 14th European Conference, ECCV 2016, Proceedings. editor / Bastian Leibe ; Jiri Matas ; Nicu Sebe ; Max Welling. Springer Verlag, 2016. pp. 484-499 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{d05014877bf641768683fa8b080f040d,

title = "Learning a predictable and generative vector representation for objects",

abstract = "What is a good vector representation of an object? We believe that it should be generative in 3D, in the sense that it can produce new 3D objects; as well as be predictable from 2D, in the sense that it can be perceived from 2D images. We propose a novel architecture, called the TL-embedding network, to learn an embedding space with these properties. The network consists of two components: (a) an autoencoder that ensures the representation is generative; and (b) a convolutional network that ensures the representation is predictable. This enables tackling a number of tasks including voxel prediction from 2D images and 3D model retrieval. Extensive experimental analysis demonstrates the usefulness and versatility of this embedding.",

author = "Rohit Girdhar and Fouhey, {David F.} and Mikel Rodriguez and Abhinav Gupta",

note = "Funding Information: This work was partially supported by Siebel Scholarship to RG, NDSEG Fellowship to DF and Bosch Young Faculty Fellowship to AG. This material is based on research partially sponsored by ONR MURI N000141010934, ONR MURI N000141612007, NSF1320083 and a gift from Google. The authors would like to thank Yahoo! and Nvidia for the compute cluster and GPU donations respectively. The authors would also like to thank Martial Hebert and Xiaolong Wang for many helpful discussions. Publisher Copyright: {\textcopyright} Springer International Publishing AG 2016.; 14th European Conference on Computer Vision, ECCV 2016 ; Conference date: 08-10-2016 Through 16-10-2016",

year = "2016",

doi = "10.1007/978-3-319-46466-4_29",

language = "English (US)",

isbn = "9783319464657",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "484--499",

editor = "Bastian Leibe and Jiri Matas and Nicu Sebe and Max Welling",

booktitle = "Computer Vision - 14th European Conference, ECCV 2016, Proceedings",

}

TY - GEN

T1 - Learning a predictable and generative vector representation for objects

AU - Girdhar, Rohit

AU - Fouhey, David F.

AU - Rodriguez, Mikel

AU - Gupta, Abhinav

N1 - Funding Information: This work was partially supported by Siebel Scholarship to RG, NDSEG Fellowship to DF and Bosch Young Faculty Fellowship to AG. This material is based on research partially sponsored by ONR MURI N000141010934, ONR MURI N000141612007, NSF1320083 and a gift from Google. The authors would like to thank Yahoo! and Nvidia for the compute cluster and GPU donations respectively. The authors would also like to thank Martial Hebert and Xiaolong Wang for many helpful discussions. Publisher Copyright: © Springer International Publishing AG 2016.

PY - 2016

Y1 - 2016

N2 - What is a good vector representation of an object? We believe that it should be generative in 3D, in the sense that it can produce new 3D objects; as well as be predictable from 2D, in the sense that it can be perceived from 2D images. We propose a novel architecture, called the TL-embedding network, to learn an embedding space with these properties. The network consists of two components: (a) an autoencoder that ensures the representation is generative; and (b) a convolutional network that ensures the representation is predictable. This enables tackling a number of tasks including voxel prediction from 2D images and 3D model retrieval. Extensive experimental analysis demonstrates the usefulness and versatility of this embedding.

AB - What is a good vector representation of an object? We believe that it should be generative in 3D, in the sense that it can produce new 3D objects; as well as be predictable from 2D, in the sense that it can be perceived from 2D images. We propose a novel architecture, called the TL-embedding network, to learn an embedding space with these properties. The network consists of two components: (a) an autoencoder that ensures the representation is generative; and (b) a convolutional network that ensures the representation is predictable. This enables tackling a number of tasks including voxel prediction from 2D images and 3D model retrieval. Extensive experimental analysis demonstrates the usefulness and versatility of this embedding.

UR - http://www.scopus.com/inward/record.url?scp=84990019747&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84990019747&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-46466-4_29

DO - 10.1007/978-3-319-46466-4_29

M3 - Conference contribution

AN - SCOPUS:84990019747

SN - 9783319464657

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 484

EP - 499

BT - Computer Vision - 14th European Conference, ECCV 2016, Proceedings

A2 - Leibe, Bastian

A2 - Matas, Jiri

A2 - Sebe, Nicu

A2 - Welling, Max

PB - Springer Verlag

T2 - 14th European Conference on Computer Vision, ECCV 2016

Y2 - 8 October 2016 through 16 October 2016

ER -

Learning a predictable and generative vector representation for objects

Abstract

Publication series

Conference

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this