Transfer of task representation in Reinforcement Learning using policy-based proto-value functions

Eliseo Ferrante, Alessandro Lazaric, Marcello Restelli

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Reinforcement Learning research is traditionally devoted to solve single-task problems. Therefore, anytime a new task is faced, learning must be restarted from scratch. Recently, several studies have addressed the issue of reusing the knowledge acquired in solving previous related tasks by transfer- ring information about policies and value functions. In this paper, we analyze the use of proto-value functions under the transfer learning perspective. Proto-value functions are effective basis functions for the approximation of value functions defined over the graph obtained by a random walk on the environment. The definition of this graph is a key aspect in transfer transfer problems in which both the reward function and the dynamics change. Therefore, we introduce policy-based proto-value functions, which can be obtained by considering the graph generated by a random walk guided by the optimal policy of one of the tasks at hand. We compare the effectiveness of policy-based and standard proto-value functions, on different transfer problems defined on a simple grid-world environment.

Original languageEnglish (US)
Title of host publication7th International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS 2008
PublisherInternational Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS)
Pages1301-1304
Number of pages4
ISBN (Print)9781605604701
StatePublished - 2008
Event7th International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS 2008 - Estoril, Portugal
Duration: May 12 2008May 16 2008

Publication series

NameProceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS
Volume3
ISSN (Print)1548-8403
ISSN (Electronic)1558-2914

Other

Other7th International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS 2008
Country/TerritoryPortugal
CityEstoril
Period5/12/085/16/08

Keywords

  • Proto-value functions
  • Reinforcement Learning
  • Transfer Learning

ASJC Scopus subject areas

  • Artificial Intelligence
  • Software
  • Control and Systems Engineering

Fingerprint

Dive into the research topics of 'Transfer of task representation in Reinforcement Learning using policy-based proto-value functions'. Together they form a unique fingerprint.

Cite this