Learning-Based Adaptive Optimal Output Regulation of Discrete-Time Linear Systems

Sayan Chakraborty; Weinan Gao; Leilei Cui; Frank L. Lewis; Zhong Ping Jiang

doi:10.1016/j.ifacol.2023.10.912

Learning-Based Adaptive Optimal Output Regulation of Discrete-Time Linear Systems

Sayan Chakraborty, Weinan Gao, Leilei Cui, Frank L. Lewis, Zhong Ping Jiang

Electrical and Computer Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

In this paper, we address the problem of model-free optimal output regulation of discrete-time systems that aims at achieving asymptotic tracking and disturbance rejection without the knowledge of the system parameters. Insights from reinforcement learning and adaptive dynamic programming are used to solve this problem. An interesting discovery is that the model-free discrete-time output regulation differs from the continuous-time counterpart in terms of the persistent excitation condition required to ensure the uniqueness and convergence of the policy iteration. In this work, we carefully establish the persistent excitation condition to ensure the uniqueness and convergence properties of the policy iteration.

Original language	English (US)
Title of host publication	IFAC-PapersOnLine
Editors	Hideaki Ishii, Yoshio Ebihara, Jun-ichi Imura, Masaki Yamakita
Publisher	Elsevier B.V.
Pages	10283-10288
Number of pages	6
Edition	2
ISBN (Electronic)	9781713872344
DOIs	https://doi.org/10.1016/j.ifacol.2023.10.912
State	Published - Jul 1 2023
Event	22nd IFAC World Congress - Yokohama, Japan Duration: Jul 9 2023 → Jul 14 2023

Publication series

Name	IFAC-PapersOnLine
Number	2
Volume	56
ISSN (Electronic)	2405-8963

Conference

Conference	22nd IFAC World Congress
Country/Territory	Japan
City	Yokohama
Period	7/9/23 → 7/14/23

Keywords

Adaptive control
approximate/adaptive dynamic programming
discrete-time output regulation
discrete-time systems
optimal control

ASJC Scopus subject areas

Control and Systems Engineering

Access to Document

10.1016/j.ifacol.2023.10.912

Cite this

Learning-Based Adaptive Optimal Output Regulation of Discrete-Time Linear Systems. / Chakraborty, Sayan; Gao, Weinan; Cui, Leilei et al.
IFAC-PapersOnLine. ed. / Hideaki Ishii; Yoshio Ebihara; Jun-ichi Imura; Masaki Yamakita. 2. ed. Elsevier B.V., 2023. p. 10283-10288 (IFAC-PapersOnLine; Vol. 56, No. 2).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Chakraborty, S, Gao, W, Cui, L, Lewis, FL & Jiang, ZP 2023, Learning-Based Adaptive Optimal Output Regulation of Discrete-Time Linear Systems. in H Ishii, Y Ebihara, J Imura & M Yamakita (eds), IFAC-PapersOnLine. 2 edn, IFAC-PapersOnLine, no. 2, vol. 56, Elsevier B.V., pp. 10283-10288, 22nd IFAC World Congress, Yokohama, Japan, 7/9/23. https://doi.org/10.1016/j.ifacol.2023.10.912

@inproceedings{d9278e19290d4b2abe74dc9561d6b9a7,

title = "Learning-Based Adaptive Optimal Output Regulation of Discrete-Time Linear Systems",

abstract = "In this paper, we address the problem of model-free optimal output regulation of discrete-time systems that aims at achieving asymptotic tracking and disturbance rejection without the knowledge of the system parameters. Insights from reinforcement learning and adaptive dynamic programming are used to solve this problem. An interesting discovery is that the model-free discrete-time output regulation differs from the continuous-time counterpart in terms of the persistent excitation condition required to ensure the uniqueness and convergence of the policy iteration. In this work, we carefully establish the persistent excitation condition to ensure the uniqueness and convergence properties of the policy iteration.",

keywords = "Adaptive control, approximate/adaptive dynamic programming, discrete-time output regulation, discrete-time systems, optimal control",

author = "Sayan Chakraborty and Weinan Gao and Leilei Cui and Lewis, {Frank L.} and Jiang, {Zhong Ping}",

note = "Publisher Copyright: Copyright {\textcopyright} 2023 The Authors. This is an open access article under the CC BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/4.0/); 22nd IFAC World Congress ; Conference date: 09-07-2023 Through 14-07-2023",

year = "2023",

month = jul,

day = "1",

doi = "10.1016/j.ifacol.2023.10.912",

language = "English (US)",

series = "IFAC-PapersOnLine",

publisher = "Elsevier B.V.",

number = "2",

pages = "10283--10288",

editor = "Hideaki Ishii and Yoshio Ebihara and Jun-ichi Imura and Masaki Yamakita",

booktitle = "IFAC-PapersOnLine",

edition = "2",

}

TY - GEN

T1 - Learning-Based Adaptive Optimal Output Regulation of Discrete-Time Linear Systems

AU - Chakraborty, Sayan

AU - Gao, Weinan

AU - Cui, Leilei

AU - Lewis, Frank L.

AU - Jiang, Zhong Ping

PY - 2023/7/1

Y1 - 2023/7/1

N2 - In this paper, we address the problem of model-free optimal output regulation of discrete-time systems that aims at achieving asymptotic tracking and disturbance rejection without the knowledge of the system parameters. Insights from reinforcement learning and adaptive dynamic programming are used to solve this problem. An interesting discovery is that the model-free discrete-time output regulation differs from the continuous-time counterpart in terms of the persistent excitation condition required to ensure the uniqueness and convergence of the policy iteration. In this work, we carefully establish the persistent excitation condition to ensure the uniqueness and convergence properties of the policy iteration.

AB - In this paper, we address the problem of model-free optimal output regulation of discrete-time systems that aims at achieving asymptotic tracking and disturbance rejection without the knowledge of the system parameters. Insights from reinforcement learning and adaptive dynamic programming are used to solve this problem. An interesting discovery is that the model-free discrete-time output regulation differs from the continuous-time counterpart in terms of the persistent excitation condition required to ensure the uniqueness and convergence of the policy iteration. In this work, we carefully establish the persistent excitation condition to ensure the uniqueness and convergence properties of the policy iteration.

KW - Adaptive control

KW - approximate/adaptive dynamic programming

KW - discrete-time output regulation

KW - discrete-time systems

KW - optimal control

UR - http://www.scopus.com/inward/record.url?scp=85183662778&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85183662778&partnerID=8YFLogxK

U2 - 10.1016/j.ifacol.2023.10.912

DO - 10.1016/j.ifacol.2023.10.912

M3 - Conference contribution

AN - SCOPUS:85183662778

T3 - IFAC-PapersOnLine

SP - 10283

EP - 10288

BT - IFAC-PapersOnLine

A2 - Ishii, Hideaki

A2 - Ebihara, Yoshio

A2 - Imura, Jun-ichi

A2 - Yamakita, Masaki

PB - Elsevier B.V.

T2 - 22nd IFAC World Congress

Y2 - 9 July 2023 through 14 July 2023

ER -

Learning-Based Adaptive Optimal Output Regulation of Discrete-Time Linear Systems

Abstract

Publication series

Conference

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this