A Continuous Information Gain Measure to Find the Most Discriminatory Problems for AI Benchmarking

Matthew Stephenson, Damien Anderson, Ahmed Khalifa, John Levine, Jochen Renz, Julian Togelius, Christoph Salge

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Abstract

    This paper introduces an information-theoretic method for selecting a subset of problems which gives the most information about a group of problem-solving algorithms. This method was tested on the games in the General Video Game AI (GVGAI) framework, allowing us to identify a smaller set of games that still gives a large amount of information about the abilities of different game-playing agents. This approach can be used to make agent testing more efficient. We can achieve almost as good discriminatory accuracy when testing on only a handful of games as when testing on more than a hundred games, something which is often computationally infeasible. Furthermore, this method can be extended to study the dimensions of the effective variance in game design between these games, allowing us to identify which games differentiate between agents in the most complementary ways.

    Original languageEnglish (US)
    Title of host publication2020 IEEE Congress on Evolutionary Computation, CEC 2020 - Conference Proceedings
    PublisherInstitute of Electrical and Electronics Engineers Inc.
    ISBN (Electronic)9781728169293
    DOIs
    StatePublished - Jul 2020
    Event2020 IEEE Congress on Evolutionary Computation, CEC 2020 - Virtual, Glasgow, United Kingdom
    Duration: Jul 19 2020Jul 24 2020

    Publication series

    Name2020 IEEE Congress on Evolutionary Computation, CEC 2020 - Conference Proceedings

    Conference

    Conference2020 IEEE Congress on Evolutionary Computation, CEC 2020
    CountryUnited Kingdom
    CityVirtual, Glasgow
    Period7/19/207/24/20

    Keywords

    • General Video Game AI
    • Information Gain

    ASJC Scopus subject areas

    • Control and Optimization
    • Decision Sciences (miscellaneous)
    • Artificial Intelligence
    • Computer Vision and Pattern Recognition
    • Hardware and Architecture

    Fingerprint Dive into the research topics of 'A Continuous Information Gain Measure to Find the Most Discriminatory Problems for AI Benchmarking'. Together they form a unique fingerprint.

    Cite this