Lagrangian Control through Deep-RL: Applications to Bottleneck Decongestion

Eugene Vinitsky, Kanaad Parvate, Aboudy Kreidieh, Cathy Wu, Alexandre Bayen

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Using deep reinforcement learning, we derive novel control policies for autonomous vehicles to improve the throughput of a bottleneck modeled after the San Francisco-Oakland Bay Bridge. Using Flow, a new library for applying deep reinforcement learning to traffic micro-simulators, we consider the problem of improving the throughput of a traffic benchmark: a two-stage bottleneck where four lanes reduce to two and then reduce to one. We first characterize the inflow-outflow curve of this bottleneck without any control. We introduce an inflow of autonomous vehicles with the intent of improving the congestion through Lagrangian control. To handle the varying number of autonomous vehicles in the system we derive a per-lane variable speed limits parametrization of the controller. We demonstrate that a 10% penetration rate of controlled autonomous vehicles can improve the throughput of the bottleneck by 200 vehicles per hour: a 25% improvement at high inflows. Finally, we compare the performance of our control policies to feedback ramp metering and show that the AV controller provides comparable performance to ramp metering without the need to build new ramp metering infrastructure. Illustrative videos of the results can be found at and code and tutorials can be found at

Original languageEnglish (US)
Title of host publication2018 IEEE Intelligent Transportation Systems Conference, ITSC 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
Number of pages7
ISBN (Electronic)9781728103235
StatePublished - Dec 7 2018
Event21st IEEE International Conference on Intelligent Transportation Systems, ITSC 2018 - Maui, United States
Duration: Nov 4 2018Nov 7 2018

Publication series

NameIEEE Conference on Intelligent Transportation Systems, Proceedings, ITSC


Conference21st IEEE International Conference on Intelligent Transportation Systems, ITSC 2018
Country/TerritoryUnited States

ASJC Scopus subject areas

  • Automotive Engineering
  • Mechanical Engineering
  • Computer Science Applications


Dive into the research topics of 'Lagrangian Control through Deep-RL: Applications to Bottleneck Decongestion'. Together they form a unique fingerprint.

Cite this