Stochastic Video Generation with a Learned Prior

Emily Denton; Rob Fergus

Stochastic Video Generation with a Learned Prior

Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Generating video frames that accurately predict future world states is challenging. Existing ap-proaches either fail to capture the full distribution of outcomes, or yield blurry generations, or both. In this paper we introduce a video generation model with a learned prior over stochastic latent variables at each time step. Video frames are generated by drawing samples from this prior and combining them with a deterministic estimate of the future frame. The approach is simple and easily trained end-to-end on a variety of datascts. Sample generations are both varied and sharp, even many frames into the future, and compare favorably to those from existing approaches.

Original language	English (US)
Title of host publication	35th International Conference on Machine Learning, ICML 2018
Editors	Andreas Krause, Jennifer Dy
Publisher	International Machine Learning Society (IMLS)
Pages	1906-1919
Number of pages	14
ISBN (Electronic)	9781510867963
State	Published - 2018
Event	35th International Conference on Machine Learning, ICML 2018 - Stockholm, Sweden Duration: Jul 10 2018 → Jul 15 2018

Publication series

Name	35th International Conference on Machine Learning, ICML 2018
Volume	3

Other

Other	35th International Conference on Machine Learning, ICML 2018
Country/Territory	Sweden
City	Stockholm
Period	7/10/18 → 7/15/18

ASJC Scopus subject areas

Computational Theory and Mathematics
Human-Computer Interaction
Software

Cite this

Stochastic Video Generation with a Learned Prior. / Denton, Emily; Fergus, Rob.
35th International Conference on Machine Learning, ICML 2018. ed. / Andreas Krause; Jennifer Dy. International Machine Learning Society (IMLS), 2018. p. 1906-1919 (35th International Conference on Machine Learning, ICML 2018; Vol. 3).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Denton, E & Fergus, R 2018, Stochastic Video Generation with a Learned Prior. in A Krause & J Dy (eds), 35th International Conference on Machine Learning, ICML 2018. 35th International Conference on Machine Learning, ICML 2018, vol. 3, International Machine Learning Society (IMLS), pp. 1906-1919, 35th International Conference on Machine Learning, ICML 2018, Stockholm, Sweden, 7/10/18.

@inproceedings{3ded21ab0f8f433b87e82afa68e743ff,

title = "Stochastic Video Generation with a Learned Prior",

abstract = "Generating video frames that accurately predict future world states is challenging. Existing ap-proaches either fail to capture the full distribution of outcomes, or yield blurry generations, or both. In this paper we introduce a video generation model with a learned prior over stochastic latent variables at each time step. Video frames are generated by drawing samples from this prior and combining them with a deterministic estimate of the future frame. The approach is simple and easily trained end-to-end on a variety of datascts. Sample generations are both varied and sharp, even many frames into the future, and compare favorably to those from existing approaches.",

author = "Emily Denton and Rob Fergus",

note = "Publisher Copyright: {\textcopyright} 2018 35th International Conference on Machine Learning, ICML 2018. All rights reserved.; 35th International Conference on Machine Learning, ICML 2018 ; Conference date: 10-07-2018 Through 15-07-2018",

year = "2018",

language = "English (US)",

series = "35th International Conference on Machine Learning, ICML 2018",

publisher = "International Machine Learning Society (IMLS)",

pages = "1906--1919",

editor = "Andreas Krause and Jennifer Dy",

booktitle = "35th International Conference on Machine Learning, ICML 2018",

}

TY - GEN

T1 - Stochastic Video Generation with a Learned Prior

AU - Denton, Emily

AU - Fergus, Rob

PY - 2018

Y1 - 2018

N2 - Generating video frames that accurately predict future world states is challenging. Existing ap-proaches either fail to capture the full distribution of outcomes, or yield blurry generations, or both. In this paper we introduce a video generation model with a learned prior over stochastic latent variables at each time step. Video frames are generated by drawing samples from this prior and combining them with a deterministic estimate of the future frame. The approach is simple and easily trained end-to-end on a variety of datascts. Sample generations are both varied and sharp, even many frames into the future, and compare favorably to those from existing approaches.

AB - Generating video frames that accurately predict future world states is challenging. Existing ap-proaches either fail to capture the full distribution of outcomes, or yield blurry generations, or both. In this paper we introduce a video generation model with a learned prior over stochastic latent variables at each time step. Video frames are generated by drawing samples from this prior and combining them with a deterministic estimate of the future frame. The approach is simple and easily trained end-to-end on a variety of datascts. Sample generations are both varied and sharp, even many frames into the future, and compare favorably to those from existing approaches.

UR - http://www.scopus.com/inward/record.url?scp=85057280496&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85057280496&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:85057280496

T3 - 35th International Conference on Machine Learning, ICML 2018

SP - 1906

EP - 1919

BT - 35th International Conference on Machine Learning, ICML 2018

A2 - Krause, Andreas

A2 - Dy, Jennifer

PB - International Machine Learning Society (IMLS)

T2 - 35th International Conference on Machine Learning, ICML 2018

Y2 - 10 July 2018 through 15 July 2018

ER -

Stochastic Video Generation with a Learned Prior

Abstract

Publication series

Other

ASJC Scopus subject areas

Other files and links

Fingerprint

Cite this