Resistive Crossbar-Aware Neural Network Design and Optimization

Muhammad Abdullah Hanif, Aditya Manglik, Muhammad Shafique

Research output: Contribution to journalArticlepeer-review

Abstract

Recent research in Non-Volatile Memory (NVM) and Processing-in-Memory (PIM) technologies has proposed low energy PIM-based system designs for high-performance neural network inference. Simultaneously, there is a tremendous thrust in neural network architecture research, primarily targeted towards task-specific accuracy improvements. Despite the enormous potential of a PIM-based compute paradigm, most hardware proposals adopt a one-accelerator-fits-all-networks approach, bleeding performance across all verticals. The overarching goal for this work is to improve the throughput and power efficiency of convolutional neural networks on resistive crossbar-based microarchitectures. To this end, we demonstrate why, how, and where to prune contemporary neural networks for superior exploitation of the crossbar’s underlying parallelism model. Further, we present the first crossbar-aware neural network design principles for discovering novel crossbar-amenable network architectures. Our third contribution includes simple yet efficient hardware optimizations to boost energy & area efficiency for modern deep neural networks and ensembles. Finally, we combine these ideas towards our fourth contribution, CrossNet, a novel network architecture family which improves computational efficiency by 19.06× and power efficiency by 4.16× over state-of-the-art designs.

Original languageEnglish (US)
JournalIEEE Access
DOIs
StateAccepted/In press - 2020

Keywords

  • CNN
  • Computational modeling
  • Computer architecture
  • Convolutional Neural Networks
  • Crossbar
  • CrossNet
  • Deep Neural Networks
  • Design
  • DNN
  • Efficiency
  • Energy Consumption
  • Ensemble
  • Hardware
  • In-Memory Computing
  • Memristor
  • Network architecture
  • Neural Network
  • Neural networks
  • Neuromorphic Computing
  • Optimization
  • Parallel processing
  • Performance
  • Principles
  • Processing-in-Memory
  • Pruning
  • ReRAM
  • Resistive RAM

ASJC Scopus subject areas

  • Computer Science(all)
  • Materials Science(all)
  • Engineering(all)

Fingerprint Dive into the research topics of 'Resistive Crossbar-Aware Neural Network Design and Optimization'. Together they form a unique fingerprint.

Cite this