Reinforcement Learning for Adaptive Periodic Linear Quadratic Control

Bo Pang; Zhong Ping Jiang; Iven Mareels

doi:10.1109/CDC40024.2019.9030252

Reinforcement Learning for Adaptive Periodic Linear Quadratic Control

Bo Pang, Zhong Ping Jiang, Iven Mareels

Electrical and Computer Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

This paper presents a first solution to the problem of adaptive LQR for continuous-time linear periodic systems. Specifically, reinforcement learning and adaptive dynamic programming (ADP) techniques are used to develop two algorithms to obtain near-optimal controllers. Firstly, the policy iteration (PI) and value iteration (VI) methods are proposed when the model is known. Then, PI-based and VI-based off-policy ADP algorithms are derived to find near-optimal solutions directly from input/state data collected along the system trajectories, without the exact knowledge of system dynamics. The effectiveness of the derived algorithms is validated using the well-known lossy Mathieu equation.

Original language	English (US)
Title of host publication	2019 IEEE 58th Conference on Decision and Control, CDC 2019
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	3322-3327
Number of pages	6
ISBN (Electronic)	9781728113982
DOIs	https://doi.org/10.1109/CDC40024.2019.9030252
State	Published - Dec 2019
Event	58th IEEE Conference on Decision and Control, CDC 2019 - Nice, France Duration: Dec 11 2019 → Dec 13 2019

Publication series

Name	Proceedings of the IEEE Conference on Decision and Control
Volume	2019-December
ISSN (Print)	0743-1546
ISSN (Electronic)	2576-2370

Conference

Conference	58th IEEE Conference on Decision and Control, CDC 2019
Country/Territory	France
City	Nice
Period	12/11/19 → 12/13/19

ASJC Scopus subject areas

Control and Systems Engineering
Modeling and Simulation
Control and Optimization

Access to Document

10.1109/CDC40024.2019.9030252

Cite this

Pang, B., Jiang, Z. P., & Mareels, I. (2019). Reinforcement Learning for Adaptive Periodic Linear Quadratic Control. In 2019 IEEE 58th Conference on Decision and Control, CDC 2019 (pp. 3322-3327). [9030252] (Proceedings of the IEEE Conference on Decision and Control; Vol. 2019-December). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/CDC40024.2019.9030252

Reinforcement Learning for Adaptive Periodic Linear Quadratic Control. / Pang, Bo; Jiang, Zhong Ping; Mareels, Iven.
2019 IEEE 58th Conference on Decision and Control, CDC 2019. Institute of Electrical and Electronics Engineers Inc., 2019. p. 3322-3327 9030252 (Proceedings of the IEEE Conference on Decision and Control; Vol. 2019-December).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Pang, B, Jiang, ZP & Mareels, I 2019, Reinforcement Learning for Adaptive Periodic Linear Quadratic Control. in 2019 IEEE 58th Conference on Decision and Control, CDC 2019., 9030252, Proceedings of the IEEE Conference on Decision and Control, vol. 2019-December, Institute of Electrical and Electronics Engineers Inc., pp. 3322-3327, 58th IEEE Conference on Decision and Control, CDC 2019, Nice, France, 12/11/19. https://doi.org/10.1109/CDC40024.2019.9030252

@inproceedings{ed041c42d96c4cc7a685666dd8fa4063,

title = "Reinforcement Learning for Adaptive Periodic Linear Quadratic Control",

abstract = "This paper presents a first solution to the problem of adaptive LQR for continuous-time linear periodic systems. Specifically, reinforcement learning and adaptive dynamic programming (ADP) techniques are used to develop two algorithms to obtain near-optimal controllers. Firstly, the policy iteration (PI) and value iteration (VI) methods are proposed when the model is known. Then, PI-based and VI-based off-policy ADP algorithms are derived to find near-optimal solutions directly from input/state data collected along the system trajectories, without the exact knowledge of system dynamics. The effectiveness of the derived algorithms is validated using the well-known lossy Mathieu equation.",

author = "Bo Pang and Jiang, {Zhong Ping} and Iven Mareels",

note = "Funding Information: *This work has been supported in part by the U.S. National Science Foundation grant ECCS-1501044. Publisher Copyright: {\textcopyright} 2019 IEEE.; 58th IEEE Conference on Decision and Control, CDC 2019 ; Conference date: 11-12-2019 Through 13-12-2019",

year = "2019",

month = dec,

doi = "10.1109/CDC40024.2019.9030252",

language = "English (US)",

series = "Proceedings of the IEEE Conference on Decision and Control",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "3322--3327",

booktitle = "2019 IEEE 58th Conference on Decision and Control, CDC 2019",

}

TY - GEN

T1 - Reinforcement Learning for Adaptive Periodic Linear Quadratic Control

AU - Pang, Bo

AU - Jiang, Zhong Ping

AU - Mareels, Iven

PY - 2019/12

Y1 - 2019/12

N2 - This paper presents a first solution to the problem of adaptive LQR for continuous-time linear periodic systems. Specifically, reinforcement learning and adaptive dynamic programming (ADP) techniques are used to develop two algorithms to obtain near-optimal controllers. Firstly, the policy iteration (PI) and value iteration (VI) methods are proposed when the model is known. Then, PI-based and VI-based off-policy ADP algorithms are derived to find near-optimal solutions directly from input/state data collected along the system trajectories, without the exact knowledge of system dynamics. The effectiveness of the derived algorithms is validated using the well-known lossy Mathieu equation.

AB - This paper presents a first solution to the problem of adaptive LQR for continuous-time linear periodic systems. Specifically, reinforcement learning and adaptive dynamic programming (ADP) techniques are used to develop two algorithms to obtain near-optimal controllers. Firstly, the policy iteration (PI) and value iteration (VI) methods are proposed when the model is known. Then, PI-based and VI-based off-policy ADP algorithms are derived to find near-optimal solutions directly from input/state data collected along the system trajectories, without the exact knowledge of system dynamics. The effectiveness of the derived algorithms is validated using the well-known lossy Mathieu equation.

UR - http://www.scopus.com/inward/record.url?scp=85082504963&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85082504963&partnerID=8YFLogxK

U2 - 10.1109/CDC40024.2019.9030252

DO - 10.1109/CDC40024.2019.9030252

M3 - Conference contribution

AN - SCOPUS:85082504963

T3 - Proceedings of the IEEE Conference on Decision and Control

SP - 3322

EP - 3327

BT - 2019 IEEE 58th Conference on Decision and Control, CDC 2019

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 58th IEEE Conference on Decision and Control, CDC 2019

Y2 - 11 December 2019 through 13 December 2019

ER -

Reinforcement Learning for Adaptive Periodic Linear Quadratic Control

Abstract

Publication series

Conference

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this