"Superstition" in the network: Deep reinforcement learning plays deceptive games

Philip Bontrager, Ahmed Khalifa, Damien Anderson, Matthew Stephenson, Christoph Salge, Julian Togelius

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Abstract

    Deep reinforcement learning has learned to play many games well, but failed on others. To better characterize the modes and reasons of failure of deep reinforcement learners, we test the widely used Asynchronous Actor-Critic (A2C) algorithm on four deceptive games, which are specially designed to provide challenges to game-playing agents. These games are implemented in the General Video Game AI framework, which allows us to compare the behavior of reinforcement learningbased agents with planning agents based on tree search. We find that several of these games reliably deceive deep reinforcement learners, and that the resulting behavior highlights the shortcomings of the learning algorithm. The particular ways in which agents fail differ from how planning-based agents fail, further illuminating the character of these algorithms. We propose an initial typology of deceptions which could help us better understand pitfalls and failure modes of (deep) reinforcement learning.

    Original languageEnglish (US)
    Title of host publicationProceedings of the 15th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, AIIDE 2019
    PublisherAAAI press
    Pages10-16
    Number of pages7
    ISBN (Electronic)9781577358190
    StatePublished - 2019
    Event15th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, AIIDE 2019 - Atlanta, United States
    Duration: Oct 8 2019Oct 12 2019

    Publication series

    NameProceedings of the 15th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, AIIDE 2019

    Conference

    Conference15th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, AIIDE 2019
    CountryUnited States
    CityAtlanta
    Period10/8/1910/12/19

    ASJC Scopus subject areas

    • Visual Arts and Performing Arts
    • Artificial Intelligence

    Fingerprint Dive into the research topics of '"Superstition" in the network: Deep reinforcement learning plays deceptive games'. Together they form a unique fingerprint.

    Cite this