Asymmetric Actor Critic for Image-Based Robot Learning

Lerrel Pinto, Marcin Andrychowicz, Peter Welinder, Wojciech Zaremba, Pieter Abbeel

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Deep reinforcement learning (RL) has proven a powerful technique in many sequential decision making domains. However, robotics poses many challenges for RL, most notably training on a physical system can be expensive and dangerous, which has sparked significant interest in learning control policies using a physics simulator. While several recent works have shown promising results in transferring policies trained in simulation to the real world, they often do not fully utilize the advantage of working with a simulator. In this work, we propose the Asymmetric Actor Critic, which learns a vision-based control policy while taking advantage of access to the underlying state to significantly speed up training. Concretely, our algorithm employs an actor-critic training algorithm in which the critic is trained on full states while the actor (or policy) is trained on images. We show that using these asymmetric inputs improves performance on a range of simulated tasks. Finally, we combine this method with domain randomization and show real robot experiments for several tasks like picking, pushing, and moving a block. We achieve this simulation to real-world transfer without training on any real-world data. Videos of these experiments can be found in

Original languageEnglish (US)
Title of host publicationRobotics
Subtitle of host publicationScience and Systems XIV
EditorsHadas Kress-Gazit, Siddhartha S. Srinivasa, Tom Howard, Nikolay Atanasov
PublisherMIT Press Journals
ISBN (Print)9780992374747
StatePublished - 2018
Event14th Robotics: Science and Systems, RSS 2018 - Pittsburgh, United States
Duration: Jun 26 2018Jun 30 2018

Publication series

NameRobotics: Science and Systems
ISSN (Electronic)2330-765X


Conference14th Robotics: Science and Systems, RSS 2018
Country/TerritoryUnited States

ASJC Scopus subject areas

  • Artificial Intelligence
  • Control and Systems Engineering
  • Electrical and Electronic Engineering


Dive into the research topics of 'Asymmetric Actor Critic for Image-Based Robot Learning'. Together they form a unique fingerprint.

Cite this