PruNet: Class-Blind Pruning Method for Deep Neural Networks

Alberto Marchisio, Muhammad Abdullah Hanif, Maurizio Martina, Muhammad Shafique

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

DNNs are highly memory and computationally intensive, due to which they are unfeasible to deploy in real time or mobile applications, where power and memory resources are scarce. Introducing sparsity in the network is a way to reduce those requirements. However, systematically employing pruning under given accuracy requirements is a challenging problem. We propose a novel methodology that iteratively applies a magnitude-based Class-Blind pruning to compress a DNN for obtaining a sparse model. It is a generic methodology and can be applied to different types of DNNs. We demonstrate that retraining after pruning is essential to restore the accuracy of the network. Experimental results show that our methodology is able to reduce the model size by around two orders of magnitude, without noticeably affecting the accuracy. It requires several iterations of pruning and retraining, but can achieve up to 190x Memory Saving Ratio (for the LeNet on the MNIST dataset) when compared to the baseline model. Similar results are also obtained for more complex networks like 91x for VGG-16 on the CIFAR100 dataset. If we combine this work with an efficient coding for sparse networks, like Compressed Sparse Column (CSC) or Compressed Sparse Row (CSR), we can obtain a reduced memory footprint. Our methodology can be complemented by other compression techniques, like weight sharing, quantization or fixed-point conversion, that allows to further reduce memory and computations.

Original languageEnglish (US)
Title of host publication2018 International Joint Conference on Neural Networks, IJCNN 2018 - Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781509060146
DOIs
StatePublished - Oct 10 2018
Event2018 International Joint Conference on Neural Networks, IJCNN 2018 - Rio de Janeiro, Brazil
Duration: Jul 8 2018Jul 13 2018

Publication series

NameProceedings of the International Joint Conference on Neural Networks
Volume2018-July

Other

Other2018 International Joint Conference on Neural Networks, IJCNN 2018
CountryBrazil
CityRio de Janeiro
Period7/8/187/13/18

ASJC Scopus subject areas

  • Software
  • Artificial Intelligence

Fingerprint Dive into the research topics of 'PruNet: Class-Blind Pruning Method for Deep Neural Networks'. Together they form a unique fingerprint.

Cite this