DYNAMICS-AWARE EMBEDDINGS

William F. Whitney; Rajat Agarwal; Kyunghyun Cho; Abhinav Gupta

DYNAMICS-AWARE EMBEDDINGS

William F. Whitney, Rajat Agarwal, Kyunghyun Cho, Abhinav Gupta

Computer Science

Research output: Contribution to conference › Paper › peer-review

Abstract

In this paper we consider self-supervised representation learning to improve sample efficiency in reinforcement learning (RL). We propose a forward prediction objective for simultaneously learning embeddings of states and action sequences. These embeddings capture the structure of the environment's dynamics, enabling efficient policy learning. We demonstrate that our action embeddings alone improve the sample efficiency and peak performance of model-free RL on control from low-dimensional states. By combining state and action embeddings, we achieve efficient learning of high-quality policies on goal-conditioned continuous control from pixel observations in only 1-2 million environment steps.

Original language	English (US)
State	Published - 2020
Event	8th International Conference on Learning Representations, ICLR 2020 - Addis Ababa, Ethiopia Duration: Apr 30 2020 → …

Conference

Conference	8th International Conference on Learning Representations, ICLR 2020
Country/Territory	Ethiopia
City	Addis Ababa
Period	4/30/20 → …

ASJC Scopus subject areas

Education
Linguistics and Language
Language and Linguistics
Computer Science Applications

Cite this

@conference{04e3feec65d64473b958b2882fa41885,

title = "DYNAMICS-AWARE EMBEDDINGS",

abstract = "In this paper we consider self-supervised representation learning to improve sample efficiency in reinforcement learning (RL). We propose a forward prediction objective for simultaneously learning embeddings of states and action sequences. These embeddings capture the structure of the environment's dynamics, enabling efficient policy learning. We demonstrate that our action embeddings alone improve the sample efficiency and peak performance of model-free RL on control from low-dimensional states. By combining state and action embeddings, we achieve efficient learning of high-quality policies on goal-conditioned continuous control from pixel observations in only 1-2 million environment steps.",

author = "Whitney, {William F.} and Rajat Agarwal and Kyunghyun Cho and Abhinav Gupta",

note = "Publisher Copyright: {\textcopyright} 2020 8th International Conference on Learning Representations, ICLR 2020. All rights reserved.; 8th International Conference on Learning Representations, ICLR 2020 ; Conference date: 30-04-2020",

year = "2020",

language = "English (US)",

}

TY - CONF

T1 - DYNAMICS-AWARE EMBEDDINGS

AU - Whitney, William F.

AU - Agarwal, Rajat

AU - Cho, Kyunghyun

AU - Gupta, Abhinav

PY - 2020

Y1 - 2020

N2 - In this paper we consider self-supervised representation learning to improve sample efficiency in reinforcement learning (RL). We propose a forward prediction objective for simultaneously learning embeddings of states and action sequences. These embeddings capture the structure of the environment's dynamics, enabling efficient policy learning. We demonstrate that our action embeddings alone improve the sample efficiency and peak performance of model-free RL on control from low-dimensional states. By combining state and action embeddings, we achieve efficient learning of high-quality policies on goal-conditioned continuous control from pixel observations in only 1-2 million environment steps.

AB - In this paper we consider self-supervised representation learning to improve sample efficiency in reinforcement learning (RL). We propose a forward prediction objective for simultaneously learning embeddings of states and action sequences. These embeddings capture the structure of the environment's dynamics, enabling efficient policy learning. We demonstrate that our action embeddings alone improve the sample efficiency and peak performance of model-free RL on control from low-dimensional states. By combining state and action embeddings, we achieve efficient learning of high-quality policies on goal-conditioned continuous control from pixel observations in only 1-2 million environment steps.

UR - http://www.scopus.com/inward/record.url?scp=85150647757&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85150647757&partnerID=8YFLogxK

M3 - Paper

AN - SCOPUS:85150647757

T2 - 8th International Conference on Learning Representations, ICLR 2020

Y2 - 30 April 2020

ER -

DYNAMICS-AWARE EMBEDDINGS

Abstract

Conference

ASJC Scopus subject areas

Other files and links

Fingerprint

Cite this