H∞ Tracking Control of Completely Unknown Continuous-Time Systems via Off-Policy Reinforcement Learning

Hamidreza Modares; Frank L. Lewis; Zhong Ping Jiang

doi:10.1109/TNNLS.2015.2441749

H_∞ Tracking Control of Completely Unknown Continuous-Time Systems via Off-Policy Reinforcement Learning

Hamidreza Modares, Frank L. Lewis, Zhong Ping Jiang

Electrical and Computer Engineering

Research output: Contribution to journal › Article › peer-review

Abstract

This paper deals with the design of an H_∞ tracking controller for nonlinear continuous-time systems with completely unknown dynamics. A general bounded L₂-gain tracking problem with a discounted performance function is introduced for the H_∞ tracking. A tracking Hamilton-Jacobi-Isaac (HJI) equation is then developed that gives a Nash equilibrium solution to the associated min-max optimization problem. A rigorous analysis of bounded L₂-gain and stability of the control solution obtained by solving the tracking HJI equation is provided. An upper-bound is found for the discount factor to assure local asymptotic stability of the tracking error dynamics. An off-policy reinforcement learning algorithm is used to learn the solution to the tracking HJI equation online without requiring any knowledge of the system dynamics. Convergence of the proposed algorithm to the solution to the tracking HJI equation is shown. Simulation examples are provided to verify the effectiveness of the proposed method.

Original language	English (US)
Article number	7132753
Pages (from-to)	2550-2562
Number of pages	13
Journal	IEEE transactions on neural networks and learning systems
Volume	26
Issue number	10
DOIs	https://doi.org/10.1109/TNNLS.2015.2441749
State	Published - Oct 1 2015

Keywords

Bounded L-gain
H tracking controller
reinforcement learning (RL)
tracking Hamilton-Jacobi-Isaac (HJI) equation

ASJC Scopus subject areas

Software
Computer Science Applications
Computer Networks and Communications
Artificial Intelligence

Access to Document

10.1109/TNNLS.2015.2441749

Cite this

@article{e3481848b4344c039f58ca86657cca68,

title = "H∞ Tracking Control of Completely Unknown Continuous-Time Systems via Off-Policy Reinforcement Learning",

abstract = "This paper deals with the design of an H∞ tracking controller for nonlinear continuous-time systems with completely unknown dynamics. A general bounded L2-gain tracking problem with a discounted performance function is introduced for the H∞ tracking. A tracking Hamilton-Jacobi-Isaac (HJI) equation is then developed that gives a Nash equilibrium solution to the associated min-max optimization problem. A rigorous analysis of bounded L2-gain and stability of the control solution obtained by solving the tracking HJI equation is provided. An upper-bound is found for the discount factor to assure local asymptotic stability of the tracking error dynamics. An off-policy reinforcement learning algorithm is used to learn the solution to the tracking HJI equation online without requiring any knowledge of the system dynamics. Convergence of the proposed algorithm to the solution to the tracking HJI equation is shown. Simulation examples are provided to verify the effectiveness of the proposed method.",

keywords = "Bounded L-gain, H tracking controller, reinforcement learning (RL), tracking Hamilton-Jacobi-Isaac (HJI) equation",

author = "Hamidreza Modares and Lewis, {Frank L.} and Jiang, {Zhong Ping}",

note = "Publisher Copyright: {\textcopyright} 2015 IEEE.",

year = "2015",

month = oct,

day = "1",

doi = "10.1109/TNNLS.2015.2441749",

language = "English (US)",

volume = "26",

pages = "2550--2562",

journal = "IEEE transactions on neural networks and learning systems",

issn = "2162-237X",

publisher = "IEEE Computational Intelligence Society",

number = "10",

}

TY - JOUR

T1 - H∞ Tracking Control of Completely Unknown Continuous-Time Systems via Off-Policy Reinforcement Learning

AU - Modares, Hamidreza

AU - Lewis, Frank L.

AU - Jiang, Zhong Ping

PY - 2015/10/1

Y1 - 2015/10/1

N2 - This paper deals with the design of an H∞ tracking controller for nonlinear continuous-time systems with completely unknown dynamics. A general bounded L2-gain tracking problem with a discounted performance function is introduced for the H∞ tracking. A tracking Hamilton-Jacobi-Isaac (HJI) equation is then developed that gives a Nash equilibrium solution to the associated min-max optimization problem. A rigorous analysis of bounded L2-gain and stability of the control solution obtained by solving the tracking HJI equation is provided. An upper-bound is found for the discount factor to assure local asymptotic stability of the tracking error dynamics. An off-policy reinforcement learning algorithm is used to learn the solution to the tracking HJI equation online without requiring any knowledge of the system dynamics. Convergence of the proposed algorithm to the solution to the tracking HJI equation is shown. Simulation examples are provided to verify the effectiveness of the proposed method.

AB - This paper deals with the design of an H∞ tracking controller for nonlinear continuous-time systems with completely unknown dynamics. A general bounded L2-gain tracking problem with a discounted performance function is introduced for the H∞ tracking. A tracking Hamilton-Jacobi-Isaac (HJI) equation is then developed that gives a Nash equilibrium solution to the associated min-max optimization problem. A rigorous analysis of bounded L2-gain and stability of the control solution obtained by solving the tracking HJI equation is provided. An upper-bound is found for the discount factor to assure local asymptotic stability of the tracking error dynamics. An off-policy reinforcement learning algorithm is used to learn the solution to the tracking HJI equation online without requiring any knowledge of the system dynamics. Convergence of the proposed algorithm to the solution to the tracking HJI equation is shown. Simulation examples are provided to verify the effectiveness of the proposed method.

KW - Bounded L-gain

KW - H tracking controller

KW - reinforcement learning (RL)

KW - tracking Hamilton-Jacobi-Isaac (HJI) equation

UR - http://www.scopus.com/inward/record.url?scp=85027942299&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85027942299&partnerID=8YFLogxK

U2 - 10.1109/TNNLS.2015.2441749

DO - 10.1109/TNNLS.2015.2441749

M3 - Article

AN - SCOPUS:85027942299

SN - 2162-237X

VL - 26

SP - 2550

EP - 2562

JO - IEEE transactions on neural networks and learning systems

JF - IEEE transactions on neural networks and learning systems

IS - 10

M1 - 7132753

ER -