Learning-Based Adaptive Optimal Control of Linear Time-Delay Systems: A Policy Iteration Approach

Leilei Cui, Bo Pang, Zhong Ping Jiang

Research output: Contribution to journalArticlepeer-review


This article studies the adaptive optimal control problem for a class of linear time-delay systems described by delay differential equations. A crucial strategy is to take advantage of recent developments in reinforcement learning and adaptive dynamic programming and develop novel methods to learn adaptive optimal controllers from finite samples of input and state data. In this article, the data-driven policy iteration (PI) is proposed to solve the infinite-dimensional algebraic Riccati equation iteratively in the absence of exact model knowledge. Interestingly, the proposed recursive PI algorithm is new in the present context of continuous-time time-delay systems, even when the model knowledge is assumed known. The efficacy of the proposed learning-based control methods is validated by means of practical applications arising from metal cutting and autonomous driving.

Original languageEnglish (US)
Pages (from-to)629-636
Number of pages8
JournalIEEE Transactions on Automatic Control
Issue number1
StatePublished - Jan 1 2024


  • Adaptive dynamic programming (ADP)
  • linear time-delay systems
  • optimal control
  • policy iteration (PI)

ASJC Scopus subject areas

  • Electrical and Electronic Engineering
  • Control and Systems Engineering
  • Computer Science Applications


Dive into the research topics of 'Learning-Based Adaptive Optimal Control of Linear Time-Delay Systems: A Policy Iteration Approach'. Together they form a unique fingerprint.

Cite this