V2X-Sim: Multi-Agent Collaborative Perception Dataset and Benchmark for Autonomous Driving

Yiming Li, Dekun Ma, Ziyan An, Zixun Wang, Yiqi Zhong, Siheng Chen, Chen Feng

Research output: Contribution to journalArticlepeer-review


Vehicle-to-everything (V2X) communication techniques enable the collaboration between vehicles and many other entities in the neighboring environment, which could fundamentally improve the perception system for autonomous driving. However, the lack of a public dataset significantly restricts the research progress of collaborative perception. To fill this gap, we present V2X-Sim, a comprehensive simulated multi-agent perception dataset for V2X-aided autonomous driving. V2X-Sim provides: (1) multi-agent sensor recordings from the road-side unit (RSU) and multiple vehicles that enable collaborative perception, (2) multi-modality sensor streams that facilitate multi-modality perception, and (3) diverse ground truths that support various perception tasks. Meanwhile, we build an open-source testbed and provide a benchmark for the state-of-the-art collaborative perception algorithms on three tasks, including detection, tracking and segmentation. V2X-Sim seeks to stimulate collaborative perception research for autonomous driving before realistic datasets become widely available.

Original languageEnglish (US)
Pages (from-to)10914-10921
Number of pages8
JournalIEEE Robotics and Automation Letters
Issue number4
StatePublished - Oct 1 2022


  • Deep learning for visual perception
  • data sets for robotic vision
  • multi-robot systems

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Biomedical Engineering
  • Human-Computer Interaction
  • Mechanical Engineering
  • Computer Vision and Pattern Recognition
  • Computer Science Applications
  • Control and Optimization
  • Artificial Intelligence


Dive into the research topics of 'V2X-Sim: Multi-Agent Collaborative Perception Dataset and Benchmark for Autonomous Driving'. Together they form a unique fingerprint.

Cite this