TY - GEN
T1 - On Convergence Rate of Adaptive Multiscale Value Function Approximation for Reinforcement Learning
AU - Li, Tao
AU - Zhu, Quanyan
N1 - Funding Information:
This research is supported in part by National Science Foundation (NSF) under grant ECCS-1847056, CNS-1544782, and SES-1541164, and in part by ARO grant W911NF1910041.
Publisher Copyright:
© 2019 IEEE.
PY - 2019/10
Y1 - 2019/10
N2 - In this paper, we propose a generic framework for devising an adaptive approximation scheme for value function approximation in reinforcement learning, which introduces multiscale approximation. The two basic ingredients are multiresolution analysis as well as tree approximation. Starting from simple refinable functions, multiresolution analysis enables us to construct a wavelet system from which the basis functions are selected adaptively, resulting in a tree structure. Furthermore, we present the convergence rate of our multiscale approximation which does not depend on the regularity of basis functions.
AB - In this paper, we propose a generic framework for devising an adaptive approximation scheme for value function approximation in reinforcement learning, which introduces multiscale approximation. The two basic ingredients are multiresolution analysis as well as tree approximation. Starting from simple refinable functions, multiresolution analysis enables us to construct a wavelet system from which the basis functions are selected adaptively, resulting in a tree structure. Furthermore, we present the convergence rate of our multiscale approximation which does not depend on the regularity of basis functions.
KW - Multiscale approximation
KW - multiresolution analysis
KW - n-term approximation
KW - reinforcement learning
KW - tree approximation
KW - wavelets
UR - http://www.scopus.com/inward/record.url?scp=85077700076&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85077700076&partnerID=8YFLogxK
U2 - 10.1109/MLSP.2019.8918816
DO - 10.1109/MLSP.2019.8918816
M3 - Conference contribution
AN - SCOPUS:85077700076
T3 - IEEE International Workshop on Machine Learning for Signal Processing, MLSP
BT - 2019 IEEE 29th International Workshop on Machine Learning for Signal Processing, MLSP 2019
PB - IEEE Computer Society
T2 - 29th IEEE International Workshop on Machine Learning for Signal Processing, MLSP 2019
Y2 - 13 October 2019 through 16 October 2019
ER -