From Sim to Real: A Pipeline for Training and Deploying Traffic Smoothing Cruise Controllers

Nathan Lichtle, Eugene Vinitsky, Matthew Nice, Rahul Bhadani, Matthew Bunting, Fangyu Wu, Benedetto Piccoli, Benjamin Seibold, Daniel B. Work, Jonathan W. Lee, Jonathan Sprinkle, Alexandre M. Bayen

Research output: Contribution to journalArticlepeer-review

Abstract

Designing and validating controllers for connected and automated vehicles to enhance traffic flow presents significant challenges, from the complexity of replicating real-world stop-and-go traffic dynamics in simulation, to the intricacies involved in transitioning from simulation to actual deployment. In this work, we present a full pipeline from data collection to controller deployment. Specifically, we collect 772 km of driving data from the I-24 in Tennessee, and use it to build a one-lane simulator, placing simulated vehicles behind real-world trajectories. Using policy-gradient methods with an asymmetric critic, we improve fuel efficiency by over 10% when simulating congested scenarios. Our comprehensive approach includes reinforcement learning for controller training, software verification, hardware validation and setup, and navigating various sim-to-real challenges. Furthermore, we analyze the controller's behavior and wave-smoothing properties, and deploy it on four Toyota Rav4's in a real-world validation experiment on the I-24. Finally, we release the driving dataset (Nice et al., 2021), the simulator and the trained controller (Lichtlé et al., 2022), to enable future benchmarking and controller design.

Original languageEnglish (US)
Pages (from-to)4490-4505
Number of pages16
JournalIEEE Transactions on Robotics
Volume40
DOIs
StatePublished - 2024

Keywords

  • Autonomous vehicle navigation
  • energy and environment-aware automation
  • intelligent transportation systems
  • reinforcement learning (RL)

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Computer Science Applications
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'From Sim to Real: A Pipeline for Training and Deploying Traffic Smoothing Cruise Controllers'. Together they form a unique fingerprint.

Cite this