Bridging the Gap Between Reinforcement Learning and Nonlinear Output-Feedback Control

The primary objective of this paper is to bridge the gap between reinforcement learning (RL) and nonlinear output-feedback control by developing a novel solution to direct adaptive optimal control with guaranteed closed-loop stability More specifically, for a broad class of nonlinear affine discrete time systems with limited output measurements, we integrate RL and advanced nonlinear control methods to devise high-fidelity direct adaptive optimal controllers from data. Under the condition of uniform observability, our original learning-based control solution begins with the reconstruction of the system state from the retrospective input and output data, akin to a deadbea observer. Subsequently, we propose value iteration algorithms to facilitate the learning of optimal output-feedback control policies and value functions leveraging measured input and output data To ensure feasibility and reliability in practice, we provide rigorous convergence proofs for the proposed learning algorithms along with the stability analysis for the closed-loop system Simulation results are presented to showcase the effectivenes of the developed methodologies, demonstrating their capability to handle the output-feedback adaptive optimal control problems of general nonlinear affine systems.