Predicting object dynamics in scenes

David F. Fouhey; C. Lawrence Zitnick

doi:10.1109/CVPR.2014.260

Predicting object dynamics in scenes

David F. Fouhey, C. Lawrence Zitnick

Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Given a static scene, a human can trivially enumerate the myriad of things that can happen next and characterize the relative likelihood of each. In the process, we make use of enormous amounts of commonsense knowledge about how the world works. In this paper, we investigate learning this commonsense knowledge from data. To overcome a lack of densely annotated spatiotemporal data, we learn from sequences of abstract images gathered using crowdsourcing. The abstract scenes provide both object location and attribute information. We demonstrate qualitatively and quantitatively that our models produce plausible scene predictions on both the abstract images, as well as natural images taken from the Internet.

Original language	English (US)
Title of host publication	Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
Publisher	IEEE Computer Society
Pages	2027-2034
Number of pages	8
ISBN (Electronic)	9781479951178, 9781479951178
DOIs	https://doi.org/10.1109/CVPR.2014.260
State	Published - Sep 24 2014
Event	27th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2014 - Columbus, United States Duration: Jun 23 2014 → Jun 28 2014

Publication series

Name	Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
ISSN (Print)	1063-6919

Other

Other	27th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2014
Country/Territory	United States
City	Columbus
Period	6/23/14 → 6/28/14

Keywords

commonsense knowledge
prediction
scene dynamics
scene understanding

ASJC Scopus subject areas

Software
Computer Vision and Pattern Recognition

Access to Document

10.1109/CVPR.2014.260

Cite this

Predicting object dynamics in scenes. / Fouhey, David F.; Zitnick, C. Lawrence.
Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE Computer Society, 2014. p. 2027-2034 6909657 (Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Fouhey, DF & Zitnick, CL 2014, Predicting object dynamics in scenes. in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition., 6909657, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, IEEE Computer Society, pp. 2027-2034, 27th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2014, Columbus, United States, 6/23/14. https://doi.org/10.1109/CVPR.2014.260

@inproceedings{590933580edc4033b7906f3a671ab300,

title = "Predicting object dynamics in scenes",

abstract = "Given a static scene, a human can trivially enumerate the myriad of things that can happen next and characterize the relative likelihood of each. In the process, we make use of enormous amounts of commonsense knowledge about how the world works. In this paper, we investigate learning this commonsense knowledge from data. To overcome a lack of densely annotated spatiotemporal data, we learn from sequences of abstract images gathered using crowdsourcing. The abstract scenes provide both object location and attribute information. We demonstrate qualitatively and quantitatively that our models produce plausible scene predictions on both the abstract images, as well as natural images taken from the Internet.",

keywords = "commonsense knowledge, prediction, scene dynamics, scene understanding",

author = "Fouhey, {David F.} and Zitnick, {C. Lawrence}",

note = "Publisher Copyright: {\textcopyright} 2014 IEEE.; 27th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2014 ; Conference date: 23-06-2014 Through 28-06-2014",

year = "2014",

month = sep,

day = "24",

doi = "10.1109/CVPR.2014.260",

language = "English (US)",

series = "Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition",

publisher = "IEEE Computer Society",

pages = "2027--2034",

booktitle = "Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition",

}

TY - GEN

T1 - Predicting object dynamics in scenes

AU - Fouhey, David F.

AU - Zitnick, C. Lawrence

PY - 2014/9/24

Y1 - 2014/9/24

N2 - Given a static scene, a human can trivially enumerate the myriad of things that can happen next and characterize the relative likelihood of each. In the process, we make use of enormous amounts of commonsense knowledge about how the world works. In this paper, we investigate learning this commonsense knowledge from data. To overcome a lack of densely annotated spatiotemporal data, we learn from sequences of abstract images gathered using crowdsourcing. The abstract scenes provide both object location and attribute information. We demonstrate qualitatively and quantitatively that our models produce plausible scene predictions on both the abstract images, as well as natural images taken from the Internet.

AB - Given a static scene, a human can trivially enumerate the myriad of things that can happen next and characterize the relative likelihood of each. In the process, we make use of enormous amounts of commonsense knowledge about how the world works. In this paper, we investigate learning this commonsense knowledge from data. To overcome a lack of densely annotated spatiotemporal data, we learn from sequences of abstract images gathered using crowdsourcing. The abstract scenes provide both object location and attribute information. We demonstrate qualitatively and quantitatively that our models produce plausible scene predictions on both the abstract images, as well as natural images taken from the Internet.

KW - commonsense knowledge

KW - prediction

KW - scene dynamics

KW - scene understanding

UR - http://www.scopus.com/inward/record.url?scp=84911394491&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84911394491&partnerID=8YFLogxK

U2 - 10.1109/CVPR.2014.260

DO - 10.1109/CVPR.2014.260

M3 - Conference contribution

AN - SCOPUS:84911394491

T3 - Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition

SP - 2027

EP - 2034

BT - Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition

PB - IEEE Computer Society

T2 - 27th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2014

Y2 - 23 June 2014 through 28 June 2014

ER -

Predicting object dynamics in scenes

Abstract

Publication series

Other

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this