TY - JOUR
T1 - From Sim to Real
T2 - A Pipeline for Training and Deploying Traffic Smoothing Cruise Controllers
AU - Lichtle, Nathan
AU - Vinitsky, Eugene
AU - Nice, Matthew
AU - Bhadani, Rahul
AU - Bunting, Matthew
AU - Wu, Fangyu
AU - Piccoli, Benedetto
AU - Seibold, Benjamin
AU - Work, Daniel B.
AU - Lee, Jonathan W.
AU - Sprinkle, Jonathan
AU - Bayen, Alexandre M.
N1 - Publisher Copyright:
© 2024 IEEE.
PY - 2024
Y1 - 2024
N2 - Designing and validating controllers for connected and automated vehicles to enhance traffic flow presents significant challenges, from the complexity of replicating real-world stop-and-go traffic dynamics in simulation, to the intricacies involved in transitioning from simulation to actual deployment. In this work, we present a full pipeline from data collection to controller deployment. Specifically, we collect 772 km of driving data from the I-24 in Tennessee, and use it to build a one-lane simulator, placing simulated vehicles behind real-world trajectories. Using policy-gradient methods with an asymmetric critic, we improve fuel efficiency by over 10% when simulating congested scenarios. Our comprehensive approach includes reinforcement learning for controller training, software verification, hardware validation and setup, and navigating various sim-to-real challenges. Furthermore, we analyze the controller's behavior and wave-smoothing properties, and deploy it on four Toyota Rav4's in a real-world validation experiment on the I-24. Finally, we release the driving dataset (Nice et al., 2021), the simulator and the trained controller (Lichtlé et al., 2022), to enable future benchmarking and controller design.
AB - Designing and validating controllers for connected and automated vehicles to enhance traffic flow presents significant challenges, from the complexity of replicating real-world stop-and-go traffic dynamics in simulation, to the intricacies involved in transitioning from simulation to actual deployment. In this work, we present a full pipeline from data collection to controller deployment. Specifically, we collect 772 km of driving data from the I-24 in Tennessee, and use it to build a one-lane simulator, placing simulated vehicles behind real-world trajectories. Using policy-gradient methods with an asymmetric critic, we improve fuel efficiency by over 10% when simulating congested scenarios. Our comprehensive approach includes reinforcement learning for controller training, software verification, hardware validation and setup, and navigating various sim-to-real challenges. Furthermore, we analyze the controller's behavior and wave-smoothing properties, and deploy it on four Toyota Rav4's in a real-world validation experiment on the I-24. Finally, we release the driving dataset (Nice et al., 2021), the simulator and the trained controller (Lichtlé et al., 2022), to enable future benchmarking and controller design.
KW - Autonomous vehicle navigation
KW - energy and environment-aware automation
KW - intelligent transportation systems
KW - reinforcement learning (RL)
UR - http://www.scopus.com/inward/record.url?scp=85204503615&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85204503615&partnerID=8YFLogxK
U2 - 10.1109/TRO.2024.3463407
DO - 10.1109/TRO.2024.3463407
M3 - Article
AN - SCOPUS:85204503615
SN - 1552-3098
VL - 40
SP - 4490
EP - 4505
JO - IEEE Transactions on Robotics
JF - IEEE Transactions on Robotics
ER -