Reinforcement-Learning-Based Cooperative Adaptive Cruise Control of Buses in the Lincoln Tunnel Corridor with Time-Varying Topology

The exclusive bus lane (XBL) is one of the most popular bus transit systems in the U.S. The Lincoln Tunnel utilizes an XBL through the tunnel in the AM peak period. This paper proposes a novel data-driven cooperative adaptive cruise control (CACC) algorithm that aims to minimize a cost function for connected and autonomous buses along the XBL. Different from existing model-based CACC algorithms, the proposed approach employs the idea of reinforcement learning, which does not rely on accurate knowledge of bus dynamics. Considering a time-varying topology, where each autonomous vehicle can only receive information from preceding vehicles that are within its communication range, a distributed controller is learned real-time by online headway, velocity, and acceleration data collected from the system trajectories. The convergence of the proposed algorithm and the stability of the closed-loop system are rigorously analyzed. The effectiveness of the proposed approach is demonstrated using a well-calibrated Paramics microscopic traffic simulation model of the XBL corridor. The simulation results show that the travel time in the autonomous version of the XBL are close to the present day travel time even when the bus volume is increased by 30%.