TY - GEN
T1 - Analyzing the robustness of general video game playing agents
AU - Perez-Liebana, Diego
AU - Samothrakis, Spyridon
AU - Togelius, Julian
AU - Schaul, Tom
AU - Lucas, Simon M.
N1 - Publisher Copyright:
© 2016 IEEE.
PY - 2016/7/2
Y1 - 2016/7/2
N2 - This paper presents a study on the robustness and variability of performance of general video game-playing agents. Agents analyzed includes those that won the different legs of the 2014 and 2015 General Video Game AI Competitions, and two sample agents distributed with its framework. Initially, these agents are run in four games and ranked according to the rules of the competition. Then, different modifications to the reward signal of the games are proposed and noise is introduced in either the actions executed by the controller, their forward model, or both. Results show that it is possible to produce a significant change in the rankings by introducing the modifications proposed here. This is an important result because it enables the set of human-authored games to be automatically expanded by adding parameter-varied versions that add information and insight into the relative strengths of the agents under test. Results also show that some controllers perform well under almost all conditions, a testament to the robustness of the GVGAI benchmark.
AB - This paper presents a study on the robustness and variability of performance of general video game-playing agents. Agents analyzed includes those that won the different legs of the 2014 and 2015 General Video Game AI Competitions, and two sample agents distributed with its framework. Initially, these agents are run in four games and ranked according to the rules of the competition. Then, different modifications to the reward signal of the games are proposed and noise is introduced in either the actions executed by the controller, their forward model, or both. Results show that it is possible to produce a significant change in the rankings by introducing the modifications proposed here. This is an important result because it enables the set of human-authored games to be automatically expanded by adding parameter-varied versions that add information and insight into the relative strengths of the agents under test. Results also show that some controllers perform well under almost all conditions, a testament to the robustness of the GVGAI benchmark.
UR - http://www.scopus.com/inward/record.url?scp=85015399275&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85015399275&partnerID=8YFLogxK
U2 - 10.1109/CIG.2016.7860430
DO - 10.1109/CIG.2016.7860430
M3 - Conference contribution
AN - SCOPUS:85015399275
T3 - IEEE Conference on Computatonal Intelligence and Games, CIG
BT - 2016 IEEE Conference on Computational Intelligence and Games, CIG 2016
PB - IEEE Computer Society
T2 - 2016 IEEE Conference on Computational Intelligence and Games, CIG 2016
Y2 - 20 September 2016 through 23 September 2016
ER -