TY - GEN
T1 - A Continuous Information Gain Measure to Find the Most Discriminatory Problems for AI Benchmarking
AU - Stephenson, Matthew
AU - Anderson, Damien
AU - Khalifa, Ahmed
AU - Levine, John
AU - Renz, Jochen
AU - Togelius, Julian
AU - Salge, Christoph
N1 - Publisher Copyright:
© 2020 IEEE.
PY - 2020/7
Y1 - 2020/7
N2 - This paper introduces an information-theoretic method for selecting a subset of problems which gives the most information about a group of problem-solving algorithms. This method was tested on the games in the General Video Game AI (GVGAI) framework, allowing us to identify a smaller set of games that still gives a large amount of information about the abilities of different game-playing agents. This approach can be used to make agent testing more efficient. We can achieve almost as good discriminatory accuracy when testing on only a handful of games as when testing on more than a hundred games, something which is often computationally infeasible. Furthermore, this method can be extended to study the dimensions of the effective variance in game design between these games, allowing us to identify which games differentiate between agents in the most complementary ways.
AB - This paper introduces an information-theoretic method for selecting a subset of problems which gives the most information about a group of problem-solving algorithms. This method was tested on the games in the General Video Game AI (GVGAI) framework, allowing us to identify a smaller set of games that still gives a large amount of information about the abilities of different game-playing agents. This approach can be used to make agent testing more efficient. We can achieve almost as good discriminatory accuracy when testing on only a handful of games as when testing on more than a hundred games, something which is often computationally infeasible. Furthermore, this method can be extended to study the dimensions of the effective variance in game design between these games, allowing us to identify which games differentiate between agents in the most complementary ways.
KW - General Video Game AI
KW - Information Gain
UR - http://www.scopus.com/inward/record.url?scp=85092071829&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85092071829&partnerID=8YFLogxK
U2 - 10.1109/CEC48606.2020.9185834
DO - 10.1109/CEC48606.2020.9185834
M3 - Conference contribution
AN - SCOPUS:85092071829
T3 - 2020 IEEE Congress on Evolutionary Computation, CEC 2020 - Conference Proceedings
BT - 2020 IEEE Congress on Evolutionary Computation, CEC 2020 - Conference Proceedings
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 2020 IEEE Congress on Evolutionary Computation, CEC 2020
Y2 - 19 July 2020 through 24 July 2020
ER -