Learning-Based Adaptive Optimal Control of Linear Time-Delay Systems: A Policy Iteration Approach

Leilei Cui, Bo Pang, Zhong Ping Jiang

Research output: Contribution to journalArticlepeer-review


This paper studies the adaptive optimal control problem for a class of linear time-delay systems described by delay differential equations (DDEs). A crucial strategy is to take advantage of recent developments in reinforcement learning (RL) and adaptive dynamic programming (ADP) and develop novel methods to learn adaptive optimal controllers from finite samples of input and state data. In this paper, the data-driven policy iteration (PI) is proposed to solve the infinite-dimensional algebraic Riccati equation (ARE) iteratively in the absence of exact model knowledge. Interestingly, the proposed recursive PI algorithm is new in the present context of continuous-time time-delay systems, even when the model knowledge is assumed known. The efficacy of the proposed learning-based control methods is validated by means of practical applications arising from metal cutting and autonomous driving.

Original languageEnglish (US)
Pages (from-to)1-8
Number of pages8
JournalIEEE Transactions on Automatic Control
StateAccepted/In press - 2023


  • Adaptive dynamic programming
  • Aerospace electronics
  • Delays
  • Heuristic algorithms
  • Mathematical models
  • Optimal control
  • Stability criteria
  • Trajectory
  • linear time-delay systems
  • optimal control
  • policy iteration

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Computer Science Applications
  • Electrical and Electronic Engineering


Dive into the research topics of 'Learning-Based Adaptive Optimal Control of Linear Time-Delay Systems: A Policy Iteration Approach'. Together they form a unique fingerprint.

Cite this