FT-ClipAct: Resilience Analysis of Deep Neural Networks and Improving their Fault Tolerance using Clipped Activation

Le Ha Hoang, Muhammad Abdullah Hanif, Muhammad Shafique

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Deep Neural Networks (DNNs) are widely being adopted for safety-critical applications, e.g., healthcare and autonomous driving. Inherently, they are considered to be highly error-tolerant. However, recent studies have shown that hardware faults that impact the parameters of a DNN (e.g., weights) can have drastic impacts on its classification accuracy. In this paper, we perform a comprehensive error resilience analysis of DNNs subjected to hardware faults (e.g., permanent faults) in the weight memory. The outcome of this analysis is leveraged to propose a novel error mitigation technique which squashes the high- intensity faulty activation values to alleviate their impact. We achieve this by replacing the unbounded activation functions with their clipped versions. We also present a method to systematically define the clipping values of the activation functions that result in increased resilience of the networks against faults. We evaluate our technique on the AlexNet and the VGG-16 DNNs trained for the CIFAR-10 dataset. The experimental results show that our mitigation technique significantly improves the resilience of the DNNs to faults. For example, the proposed technique offers on average 68.92% improvement in the classification accuracy of resilience-optimized VGG-16 model at 1 × 10-5 fault rate, when compared to the base network without any fault mitigation.

Original languageEnglish (US)
Title of host publicationProceedings of the 2020 Design, Automation and Test in Europe Conference and Exhibition, DATE 2020
EditorsGiorgio Di Natale, Cristiana Bolchini, Elena-Ioana Vatajelu
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1241-1246
Number of pages6
ISBN (Electronic)9783981926347
DOIs
StatePublished - Mar 2020
Event2020 Design, Automation and Test in Europe Conference and Exhibition, DATE 2020 - Grenoble, France
Duration: Mar 9 2020Mar 13 2020

Publication series

NameProceedings of the 2020 Design, Automation and Test in Europe Conference and Exhibition, DATE 2020

Conference

Conference2020 Design, Automation and Test in Europe Conference and Exhibition, DATE 2020
CountryFrance
CityGrenoble
Period3/9/203/13/20

Keywords

  • DNN
  • Error Mitigation
  • Fault-Tolerance
  • Machine Learning
  • Reliability
  • Resilience
  • System-Level Optimization

ASJC Scopus subject areas

  • Hardware and Architecture
  • Safety, Risk, Reliability and Quality
  • Modeling and Simulation
  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'FT-ClipAct: Resilience Analysis of Deep Neural Networks and Improving their Fault Tolerance using Clipped Activation'. Together they form a unique fingerprint.

Cite this