Playing atari with six neurons (extended abstract)

Giuseppe Cuccu, Julian Togelius, Philippe Cudré-Mauroux

    Research output: Chapter in Book/Report/Conference proceedingConference contribution


    Deep reinforcement learning applied to vision-based problems like Atari games maps pixels directly to actions; internally, the deep neural network bears the responsibility of both extracting useful information and making decisions based on it. By separating image processing from decision-making, one could better understand the complexity of each task, as well as potentially find smaller policy representations that are easier for humans to understand and may generalize better. To this end, we propose a new method for learning policies and compact state representations separately but simultaneously for policy approximation in reinforcement learning. State representations are generated by an encoder based on two novel algorithms: Increasing Dictionary Vector Quantization makes the encoder capable of growing its dictionary size over time, to address new observations, and Direct Residuals Sparse Coding encodes observations by aiming for highest information inclusion. We test our system on a selection of Atari games using tiny neural networks of only 6 to 18 neurons (depending on the game's controls). These are still capable of achieving results comparable-and occasionally superior-to state-of-the-art techniques which use two orders of magnitude more neurons.

    Original languageEnglish (US)
    Title of host publicationProceedings of the 29th International Joint Conference on Artificial Intelligence, IJCAI 2020
    EditorsChristian Bessiere
    PublisherInternational Joint Conferences on Artificial Intelligence
    Number of pages5
    ISBN (Electronic)9780999241165
    StatePublished - 2020
    Event29th International Joint Conference on Artificial Intelligence, IJCAI 2020 - Yokohama, Japan
    Duration: Jan 1 2021 → …

    Publication series

    NameIJCAI International Joint Conference on Artificial Intelligence
    ISSN (Print)1045-0823


    Conference29th International Joint Conference on Artificial Intelligence, IJCAI 2020
    Period1/1/21 → …

    ASJC Scopus subject areas

    • Artificial Intelligence


    Dive into the research topics of 'Playing atari with six neurons (extended abstract)'. Together they form a unique fingerprint.

    Cite this