Development of people mass movement simulation framework based on reinforcement learning

Yanbo Pang, Takehiro Kashiyama, Takahiro Yabe, Kota Tsubouchi, Yoshihide Sekimoto

Research output: Contribution to journalArticlepeer-review


Understanding individual and crowd dynamics in urban environments is critical for numerous applications, such as urban planning, traffic forecasting, and location-based services. However, researchers have developed travel demand models to accomplish this task with survey data that are expensive and acquired at low frequencies. In contrast, emerging data collection methods have enabled researchers to leverage machine learning techniques with a tremendous amount of mobility data for analyzing and forecasting people's behaviors. In this study, we developed a reinforcement learning-based approach for modeling and simulation of people mass movement using the global positioning system (GPS) data. Unlike traditional travel demand modeling approaches, our method focuses on the problem of inferring the spatio-temporal preferences of individuals from the observed trajectories, and is based on inverse reinforcement learning (IRL) techniques. We applied the model to the data collected from a smartphone application and attempted to replicate a large amount of the population's daily movement by incorporating with agent-based multi-modal traffic simulation technologies. The simulation results indicate that agents can successfully learn and generate human-like travel activities. Furthermore, the proposed model performance significantly outperforms the existing methods in synthetic urban dynamics.

Original languageEnglish (US)
Article number102706
JournalTransportation Research Part C: Emerging Technologies
StatePublished - Aug 2020


  • Citywide people mass movement simulation
  • Mobility data
  • Reinforcement learning
  • Travel demand modeling

ASJC Scopus subject areas

  • Civil and Structural Engineering
  • Automotive Engineering
  • Transportation
  • Management Science and Operations Research


Dive into the research topics of 'Development of people mass movement simulation framework based on reinforcement learning'. Together they form a unique fingerprint.

Cite this