Self-Adaptive Driving in Nonstationary Environments through Conjectural Online Lookahead Adaptation

Tao Li, Haozhe Lei, Quanyan Zhu

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Powered by deep representation learning, re-inforcement learning (RL) provides an end-to-end learning framework capable of solving self-driving (SD) tasks without manual designs. However, time-varying nonstationary environments cause proficient but specialized RL policies to fail at execution time. For example, an RL-based SD policy trained under sunny days does not generalize well to rainy weather. Even though meta learning enables the RL agent to adapt to new tasks/environments, its offline operation fails to equip the agent with online adaptation ability when facing nonstationary environments. This work proposes an online meta reinforcement learning algorithm based on the conjectural online lookahead adaptation (COLA). COLA determines the online adaptation at every step by maximizing the agent's conjecture of the future performance in a lookahead horizon. Experimental results demonstrate that under dynamically changing weather and lighting conditions, the COLA-based self-adaptive driving outperforms the baseline policies regarding online adaptability. A demo video, source code, and appendixes are available at

Original languageEnglish (US)
Title of host publicationProceedings - ICRA 2023
Subtitle of host publicationIEEE International Conference on Robotics and Automation
PublisherInstitute of Electrical and Electronics Engineers Inc.
Number of pages7
ISBN (Electronic)9798350323658
StatePublished - 2023
Event2023 IEEE International Conference on Robotics and Automation, ICRA 2023 - London, United Kingdom
Duration: May 29 2023Jun 2 2023

Publication series

NameProceedings - IEEE International Conference on Robotics and Automation
ISSN (Print)1050-4729


Conference2023 IEEE International Conference on Robotics and Automation, ICRA 2023
Country/TerritoryUnited Kingdom

ASJC Scopus subject areas

  • Software
  • Control and Systems Engineering
  • Electrical and Electronic Engineering
  • Artificial Intelligence

Cite this