H optimal control of unknown linear discrete-time systems: An off-policy reinforcement learning approach

Bahare Kiumarsi, Hamidreza Modares, Frank L. Lewis, Zhong Ping Jiang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

This paper proposes a model-free H control design for linear discrete-time systems using reinforcement learning (RL). A novel off-policy RL algorithm is used to solve the game algebraic Riccati equation (GARE) online using the measured data along the system trajectories. The proposed RL algorithm has the following advantages compared to existing model-free RL methods for solving H control problem: 1) It is data efficient and fast since a stream of experiences which is obtained from executing a fixed behavioral policy is reused to update many value functions correspond to different leaning policies sequentially. 2) The disturbance input does not need to be adjusted in a specific manner. 3) There is no bias as a result of adding a probing noise to the control input to maintain persistence of excitation conditions. A simulation example is used to verify the effectiveness of the proposed control scheme.

Original languageEnglish (US)
Title of host publicationProceedings of the 2015 7th IEEE International Conference on Cybernetics and Intelligent Systems, CIS 2015 and Robotics, Automation and Mechatronics, RAM 2015
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages41-46
Number of pages6
ISBN (Electronic)9781467373364
DOIs
StatePublished - Sep 23 2015
Event7th IEEE International Conference on Cybernetics and Intelligent Systems, CIS 2015 and the 7th IEEE International Conference on Robotics, Automation and Mechatronics, RAM 2015 - Siem Reap, Cambodia
Duration: Jul 15 2015Jul 17 2015

Publication series

NameProceedings of the 2015 7th IEEE International Conference on Cybernetics and Intelligent Systems, CIS 2015 and Robotics, Automation and Mechatronics, RAM 2015

Other

Other7th IEEE International Conference on Cybernetics and Intelligent Systems, CIS 2015 and the 7th IEEE International Conference on Robotics, Automation and Mechatronics, RAM 2015
CountryCambodia
CitySiem Reap
Period7/15/157/17/15

Keywords

  • H control
  • game algebraic Riccati equation
  • off-policy
  • reinforcement learning

ASJC Scopus subject areas

  • Artificial Intelligence
  • Control and Systems Engineering

Fingerprint Dive into the research topics of 'H<sub>∞</sub> optimal control of unknown linear discrete-time systems: An off-policy reinforcement learning approach'. Together they form a unique fingerprint.

Cite this