Learning-Based Real-Time Process-Aware Anomaly Monitoring for Assured Autonomy

Naman Patel, Apoorva Nandini Saridena, Anna Choromanska, Prashanth Krishnamurthy, Farshad Khorrami

Research output: Contribution to journalArticlepeer-review


Various learning-based approaches have been proposed recently for robot control in complex unknown environments. However, ensuring safety in these systems is essential. Learning-based approaches are usually black-box systems which are difficult to interpret/verify. We propose a real-time continuous safety checking system which monitors the validity of the mapping from the sensor input to the actuator command (i.e., controller-focused anomaly detection) and from the actuator command to the sensor input (i.e., system-focused anomaly detection). Our controller-focused anomaly monitor is an image conditioned energy based generative adversarial network in which the discriminator acts as an energy function attributing low energies to proper actuator commands and high energy to anomalous actuator commands. Our system-focused anomaly monitor is based on an action condition video prediction framework, which finds the action conditioned future frame most similar to the actual future frame and compares it with the true action to detect anomaly. We experimentally demonstrate the effectiveness of our framework for an autonomous navigation system in indoor (on datasets from our ground vehicle) and outdoor environments (on Udacity dataset). Furthermore, we show through experimental evaluations that our proposed framework is resilient to various data corruptions common in a vision based system.

Original languageEnglish (US)
JournalIEEE Transactions on Intelligent Vehicles
StateAccepted/In press - 2020


  • Actuators
  • Anomaly detection
  • Computer architecture
  • Convolution
  • Monitoring
  • Real-time systems
  • Robot sensing systems
  • Robot vision
  • Safety and human factors
  • Safety and security
  • Vision-based systems for intelligent vehicles

ASJC Scopus subject areas

  • Artificial Intelligence
  • Automotive Engineering
  • Control and Optimization

Fingerprint Dive into the research topics of 'Learning-Based Real-Time Process-Aware Anomaly Monitoring for Assured Autonomy'. Together they form a unique fingerprint.

Cite this