Deep Reinforcement Learning for General Video Game AI

Ruben Rodriguez Torrado; Philip Bontrager; Julian Togelius; Jialin Liu; Diego Perez-Liebana

doi:10.1109/CIG.2018.8490422

Deep Reinforcement Learning for General Video Game AI

Ruben Rodriguez Torrado, Philip Bontrager, Julian Togelius, Jialin Liu, Diego Perez-Liebana

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

The General Video Game AI (GVGAI) competition and its associated software framework provides a way of benchmarking AI algorithms on a large number of games written in a domain-specific description language. While the competition has seen plenty of interest, it has so far focused on online planning, providing a forward model that allows the use of algorithms such as Monte Carlo Tree Search. In this paper, we describe how we interface GVGAI to the OpenAI Gym environment, a widely used way of connecting agents to reinforcement learning problems. Using this interface, we characterize how widely used implementations of several deep reinforcement learning algorithms fare on a number of GVGAI games. We further analyze the results to provide a first indication of the relative difficulty of these games relative to each other, and relative to those in the Arcade Learning Environment under similar conditions.

Original language	English (US)
Title of host publication	Proceedings of the 2018 IEEE Conference on Computational Intelligence and Games, CIG 2018
Publisher	IEEE Computer Society
ISBN (Electronic)	9781538643594
DOIs	https://doi.org/10.1109/CIG.2018.8490422
State	Published - Oct 11 2018
Event	14th IEEE Conference on Computational Intelligence and Games, CIG 2018 - Maastricht, Netherlands Duration: Aug 14 2018 → Aug 17 2018

Publication series

Name	IEEE Conference on Computatonal Intelligence and Games, CIG
Volume	2018-August
ISSN (Print)	2325-4270
ISSN (Electronic)	2325-4289

Other

Other	14th IEEE Conference on Computational Intelligence and Games, CIG 2018
Country/Territory	Netherlands
City	Maastricht
Period	8/14/18 → 8/17/18

Keywords

Advantage actor critic
Deep Q-learning
Deep reinforcement learning
General video game AI
OpenAI Gym
Video game description language

ASJC Scopus subject areas

Artificial Intelligence
Computer Graphics and Computer-Aided Design
Computer Vision and Pattern Recognition
Human-Computer Interaction
Software

Access to Document

10.1109/CIG.2018.8490422

Cite this

Torrado, R. R., Bontrager, P., Togelius, J., Liu, J., & Perez-Liebana, D. (2018). Deep Reinforcement Learning for General Video Game AI. In Proceedings of the 2018 IEEE Conference on Computational Intelligence and Games, CIG 2018 Article 8490422 (IEEE Conference on Computatonal Intelligence and Games, CIG; Vol. 2018-August). IEEE Computer Society. https://doi.org/10.1109/CIG.2018.8490422

Deep Reinforcement Learning for General Video Game AI. / Torrado, Ruben Rodriguez; Bontrager, Philip; Togelius, Julian et al.
Proceedings of the 2018 IEEE Conference on Computational Intelligence and Games, CIG 2018. IEEE Computer Society, 2018. 8490422 (IEEE Conference on Computatonal Intelligence and Games, CIG; Vol. 2018-August).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Torrado, RR, Bontrager, P, Togelius, J, Liu, J & Perez-Liebana, D 2018, Deep Reinforcement Learning for General Video Game AI. in Proceedings of the 2018 IEEE Conference on Computational Intelligence and Games, CIG 2018., 8490422, IEEE Conference on Computatonal Intelligence and Games, CIG, vol. 2018-August, IEEE Computer Society, 14th IEEE Conference on Computational Intelligence and Games, CIG 2018, Maastricht, Netherlands, 8/14/18. https://doi.org/10.1109/CIG.2018.8490422

@inproceedings{4a31cc9c4bb7474ab25d8025799bd3cc,

title = "Deep Reinforcement Learning for General Video Game AI",

abstract = "The General Video Game AI (GVGAI) competition and its associated software framework provides a way of benchmarking AI algorithms on a large number of games written in a domain-specific description language. While the competition has seen plenty of interest, it has so far focused on online planning, providing a forward model that allows the use of algorithms such as Monte Carlo Tree Search. In this paper, we describe how we interface GVGAI to the OpenAI Gym environment, a widely used way of connecting agents to reinforcement learning problems. Using this interface, we characterize how widely used implementations of several deep reinforcement learning algorithms fare on a number of GVGAI games. We further analyze the results to provide a first indication of the relative difficulty of these games relative to each other, and relative to those in the Arcade Learning Environment under similar conditions.",

keywords = "Advantage actor critic, Deep Q-learning, Deep reinforcement learning, General video game AI, OpenAI Gym, Video game description language",

author = "Torrado, {Ruben Rodriguez} and Philip Bontrager and Julian Togelius and Jialin Liu and Diego Perez-Liebana",

note = "Funding Information: ACKNOWLEDGEMENT This work was supported by the Ministry of Science and Technology of China (2017YFC0804003). (*) The first two authors contributed equally to this work. Publisher Copyright: {\textcopyright} 2018 IEEE.; 14th IEEE Conference on Computational Intelligence and Games, CIG 2018 ; Conference date: 14-08-2018 Through 17-08-2018",

year = "2018",

month = oct,

day = "11",

doi = "10.1109/CIG.2018.8490422",

language = "English (US)",

series = "IEEE Conference on Computatonal Intelligence and Games, CIG",

publisher = "IEEE Computer Society",

booktitle = "Proceedings of the 2018 IEEE Conference on Computational Intelligence and Games, CIG 2018",

}

TY - GEN

T1 - Deep Reinforcement Learning for General Video Game AI

AU - Torrado, Ruben Rodriguez

AU - Bontrager, Philip

AU - Togelius, Julian

AU - Liu, Jialin

AU - Perez-Liebana, Diego

N1 - Funding Information: ACKNOWLEDGEMENT This work was supported by the Ministry of Science and Technology of China (2017YFC0804003). (*) The first two authors contributed equally to this work. Publisher Copyright: © 2018 IEEE.

PY - 2018/10/11

Y1 - 2018/10/11

N2 - The General Video Game AI (GVGAI) competition and its associated software framework provides a way of benchmarking AI algorithms on a large number of games written in a domain-specific description language. While the competition has seen plenty of interest, it has so far focused on online planning, providing a forward model that allows the use of algorithms such as Monte Carlo Tree Search. In this paper, we describe how we interface GVGAI to the OpenAI Gym environment, a widely used way of connecting agents to reinforcement learning problems. Using this interface, we characterize how widely used implementations of several deep reinforcement learning algorithms fare on a number of GVGAI games. We further analyze the results to provide a first indication of the relative difficulty of these games relative to each other, and relative to those in the Arcade Learning Environment under similar conditions.

AB - The General Video Game AI (GVGAI) competition and its associated software framework provides a way of benchmarking AI algorithms on a large number of games written in a domain-specific description language. While the competition has seen plenty of interest, it has so far focused on online planning, providing a forward model that allows the use of algorithms such as Monte Carlo Tree Search. In this paper, we describe how we interface GVGAI to the OpenAI Gym environment, a widely used way of connecting agents to reinforcement learning problems. Using this interface, we characterize how widely used implementations of several deep reinforcement learning algorithms fare on a number of GVGAI games. We further analyze the results to provide a first indication of the relative difficulty of these games relative to each other, and relative to those in the Arcade Learning Environment under similar conditions.

KW - Advantage actor critic

KW - Deep Q-learning

KW - Deep reinforcement learning

KW - General video game AI

KW - OpenAI Gym

KW - Video game description language

UR - http://www.scopus.com/inward/record.url?scp=85056836621&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85056836621&partnerID=8YFLogxK

U2 - 10.1109/CIG.2018.8490422

DO - 10.1109/CIG.2018.8490422

M3 - Conference contribution

AN - SCOPUS:85056836621

T3 - IEEE Conference on Computatonal Intelligence and Games, CIG

BT - Proceedings of the 2018 IEEE Conference on Computational Intelligence and Games, CIG 2018

PB - IEEE Computer Society

T2 - 14th IEEE Conference on Computational Intelligence and Games, CIG 2018

Y2 - 14 August 2018 through 17 August 2018

ER -

Deep Reinforcement Learning for General Video Game AI

Abstract

Publication series

Other

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this