Playing atari with six neurons (extended abstract)

Giuseppe Cuccu, Julian Togelius, Philippe Cudré-Mauroux

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Abstract

    Deep reinforcement learning applied to vision-based problems like Atari games maps pixels directly to actions; internally, the deep neural network bears the responsibility of both extracting useful information and making decisions based on it. By separating image processing from decision-making, one could better understand the complexity of each task, as well as potentially find smaller policy representations that are easier for humans to understand and may generalize better. To this end, we propose a new method for learning policies and compact state representations separately but simultaneously for policy approximation in reinforcement learning. State representations are generated by an encoder based on two novel algorithms: Increasing Dictionary Vector Quantization makes the encoder capable of growing its dictionary size over time, to address new observations, and Direct Residuals Sparse Coding encodes observations by aiming for highest information inclusion. We test our system on a selection of Atari games using tiny neural networks of only 6 to 18 neurons (depending on the game's controls). These are still capable of achieving results comparable-and occasionally superior-to state-of-the-art techniques which use two orders of magnitude more neurons.

    Original languageEnglish (US)
    Title of host publicationProceedings of the 29th International Joint Conference on Artificial Intelligence, IJCAI 2020
    EditorsChristian Bessiere
    PublisherInternational Joint Conferences on Artificial Intelligence
    Pages4711-4715
    Number of pages5
    ISBN (Electronic)9780999241165
    StatePublished - 2020
    Event29th International Joint Conference on Artificial Intelligence, IJCAI 2020 - Yokohama, Japan
    Duration: Jan 1 2021 → …

    Publication series

    NameIJCAI International Joint Conference on Artificial Intelligence
    Volume2021-January
    ISSN (Print)1045-0823

    Conference

    Conference29th International Joint Conference on Artificial Intelligence, IJCAI 2020
    CountryJapan
    CityYokohama
    Period1/1/21 → …

    ASJC Scopus subject areas

    • Artificial Intelligence

    Fingerprint Dive into the research topics of 'Playing atari with six neurons (extended abstract)'. Together they form a unique fingerprint.

    Cite this