Combined Longitudinal and Lateral Control of Autonomous Vehicles based on Reinforcement Learning

In this paper, in order for the autonomous vehicle to keep a desired distance from the preceding vehicle and stay in the lane, a data-driven optimal control approach is proposed. Firstly, the dynamics of the autonomous vehicle is derived. In order to overcome the cutting-edge limitation, a virtual preceding vehicle is defined which is perpendicular to the preceding vehicle. The tracking error is defined as the deviation between the look ahead point of the autonomous vehicle and the virtual preceding vehicle. Then, the error system is derived. Secondly, based on the error system, in order to minimize the cost determined by the tracking error and the energy consumption, the Hamilton-Jacobi-Bellman (HJB) equation is established. A model-based policy iteration technique is proposed to solve the HJB equation. Thirdly, a two-phase data-driven policy iteration algorithm is proposed and implemented by using adaptive dynamic programming (ADP). The efficacy of the proposed data-driven optimal control approach is validated by computer simulations.