Playing atari with six neurons

Giuseppe Cuccu, Julian Togelius, Philippe Cudre-Mauroux

    Research output: Chapter in Book/Report/Conference proceedingConference contribution


    Deep reinforcement learning, applied to vision-based problems like Atari games, maps pixels directly to actions; internally, the deep neural network bears the responsibility of both extracting useful information and making decisions based on it By separating the image processing from decision-making, one could better understand the complexity of each task, as well as potentially find smaller policy representations that are easier for humans to understand and may generalize better To this end, we propose a new method for learning policies and compact state representations separately but simultaneously for policy approximation in reinforcement learning State representations are generated by an encoder based on two novel algorithms: Increasing Dictionary Vector Quantization makes the encoder capable of growing its dictionary size over time, to address new observations as they appear in an open-ended online-learning context; Direct Residuals Sparse Coding encodes observations by disregarding reconstruction error minimization, and aiming instead for highest information inclusion The encoder autonomously selects observations online to train on, in order to maximize code sparsity As the dictionary size increases, the encoder produces increasingly larger inputs for the neural network: This is addressed by a variation of the Exponential Natural Evolution Strategies algorithm which adapts its probability distribution dimensionality along the run We test our system on a selection of Atari games using tiny neural networks of only 6 to 18 neurons (depending on the game's controls) These are still capable of achieving results comparable-and occasionally superior-to state-of-the-art techniques which use two orders of magnitude more neurons.

    Original languageEnglish (US)
    Title of host publication18th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2019
    PublisherInternational Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS)
    Number of pages9
    ISBN (Electronic)9781510892002
    StatePublished - 2019
    Event18th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2019 - Montreal, Canada
    Duration: May 13 2019May 17 2019

    Publication series

    NameProceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS
    ISSN (Print)1548-8403
    ISSN (Electronic)1558-2914


    Conference18th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2019


    • Evolutionary algorithms
    • Game playing
    • Learning agent capabilities
    • Neuroevolution

    ASJC Scopus subject areas

    • Artificial Intelligence
    • Software
    • Control and Systems Engineering


    Dive into the research topics of 'Playing atari with six neurons'. Together they form a unique fingerprint.

    Cite this