Playing atari with six neurons

Giuseppe Cuccu, Julian Togelius, Philippe Cudre-Mauroux

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Abstract

    Deep reinforcement learning, applied to vision-based problems like Atari games, maps pixels directly to actions; internally, the deep neural network bears the responsibility of both extracting useful information and making decisions based on it By separating the image processing from decision-making, one could better understand the complexity of each task, as well as potentially find smaller policy representations that are easier for humans to understand and may generalize better To this end, we propose a new method for learning policies and compact state representations separately but simultaneously for policy approximation in reinforcement learning State representations are generated by an encoder based on two novel algorithms: Increasing Dictionary Vector Quantization makes the encoder capable of growing its dictionary size over time, to address new observations as they appear in an open-ended online-learning context; Direct Residuals Sparse Coding encodes observations by disregarding reconstruction error minimization, and aiming instead for highest information inclusion The encoder autonomously selects observations online to train on, in order to maximize code sparsity As the dictionary size increases, the encoder produces increasingly larger inputs for the neural network: This is addressed by a variation of the Exponential Natural Evolution Strategies algorithm which adapts its probability distribution dimensionality along the run We test our system on a selection of Atari games using tiny neural networks of only 6 to 18 neurons (depending on the game's controls) These are still capable of achieving results comparable-and occasionally superior-to state-of-the-art techniques which use two orders of magnitude more neurons.

    Original languageEnglish (US)
    Title of host publication18th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2019
    PublisherInternational Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS)
    Pages998-1006
    Number of pages9
    ISBN (Electronic)9781510892002
    StatePublished - 2019
    Event18th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2019 - Montreal, Canada
    Duration: May 13 2019May 17 2019

    Publication series

    NameProceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS
    Volume2
    ISSN (Print)1548-8403
    ISSN (Electronic)1558-2914

    Conference

    Conference18th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2019
    CountryCanada
    CityMontreal
    Period5/13/195/17/19

    Keywords

    • Evolutionary algorithms
    • Game playing
    • Learning agent capabilities
    • Neuroevolution

    ASJC Scopus subject areas

    • Artificial Intelligence
    • Software
    • Control and Systems Engineering

    Fingerprint Dive into the research topics of 'Playing atari with six neurons'. Together they form a unique fingerprint.

  • Cite this

    Cuccu, G., Togelius, J., & Cudre-Mauroux, P. (2019). Playing atari with six neurons. In 18th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2019 (pp. 998-1006). (Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS; Vol. 2). International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS).