DNN Patching: Progressive Fixing and Augmenting the Functionalities of DNNs for Autonomous Vehicles

Apoorva Nandini Saridena, Anna Choromanska

Research output: Contribution to journalArticlepeer-review


Autonomous vehicles have become one of the most-awaited technologies of the century. In the recent years the advances in the area of deep learning and artificial intelligence enabled rapid development of the autonomous driving sector. Some car companies have already deployed autonomous driving software into their vehicles and millions of customers are using them. Although the autonomous driving modules are typically extensively tested prior to its release on the market, the customers can either still encounter certain previously overlooked failure cases or identify missing functionalities of the model. In this letter we focus on developing the methodology for repairing the deep neural network (DNN) steering the self-driving vehicle in response to these two scenarios via patching such that the following conditions are satisfied: i) the patch is added on the top of the original DNN to extend or fix its functionality for the particular failure case, ii) the patched model keeps the original functionalities of the DNN unchanged, and iii) the size of the patch is significantly smaller than the size of the original DNN. Our approach avoids training and evaluating the network from the scratch to improve or fix it, effectively enabling fast response of the autonomous vehicle manufacturers to the customer needs. We demonstrate the plausibility of the proposed approach on the task of autonomous driving in the Carla simulator A. Dosovitskiy et al.

Original languageEnglish (US)
Pages (from-to)3257-3264
Number of pages8
JournalIEEE Robotics and Automation Letters
Issue number2
StatePublished - Apr 1 2022


  • AI-based methods
  • autonomous agents
  • continual learning

ASJC Scopus subject areas

  • Mechanical Engineering
  • Control and Optimization
  • Artificial Intelligence
  • Human-Computer Interaction
  • Control and Systems Engineering
  • Computer Vision and Pattern Recognition
  • Biomedical Engineering
  • Computer Science Applications


Dive into the research topics of 'DNN Patching: Progressive Fixing and Augmenting the Functionalities of DNNs for Autonomous Vehicles'. Together they form a unique fingerprint.

Cite this