FastCaps: A Design Methodology for Accelerating Capsule Network on Field Programmable Gate Arrays

Abdul Rahoof, Vivek Chaturvedi, Muhammad Shafique

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Capsule Network (CapsNet) has shown significant improvement in understanding the variation in images along with better generalization ability compared to traditional Convolutional Neural Network (CNN). CapsNet preserves spatial relationship among extracted features and apply dynamic routing to efficiently learn the internal connections between capsules. However, due to the capsule structure and the complexity of the routing mechanism, it is non-trivial to accelerate CapsNet performance in its original form on Field Programmable Gate Array (FPGA). Most of the existing works on CapsNet have achieved limited acceleration as they implement only the dynamic routing algorithm on FPGA, while considering all the processing steps synergistically is important for real-world applications of Capsule Networks. Towards this, we propose a novel two-step approach that deploys a full-fledged CapsNet on FPGA. First, we prune the network using a novel Look-Ahead Kernel Pruning (LAKP) methodology that uses the sum of look-ahead scores of the model parameters. Next, we simplify the nonlinear operations, reorder loops, and parallelize operations of the routing algorithm to reduce CapsNet hardware complexity. To the best of our knowledge, this is the first work accelerating a full-fledged CapsNet on FPGA. Experimental results on the MNIST and F-MNIST datasets (typical in Capsule Network community) show that the proposed LAKP approach achieves an effective compression rate of 99.26% and 98.84%, and achieves a throughput of 82 FPS and 48 FPS on Xilinx PYNQ-Z1 FPGA, respectively. Furthermore, reducing the hardware complexity of the routing algorithm increases the throughput to 1351 FPS and 934 FPS respectively. As corroborated by our results, this work enables highly performance-efficient deployment of CapsNets on low-cost FPGA that are popular in modern edge devices.

Original languageEnglish (US)
Title of host publicationIJCNN 2023 - International Joint Conference on Neural Networks, Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781665488679
DOIs
StatePublished - 2023
Event2023 International Joint Conference on Neural Networks, IJCNN 2023 - Gold Coast, Australia
Duration: Jun 18 2023Jun 23 2023

Publication series

NameProceedings of the International Joint Conference on Neural Networks
Volume2023-June

Conference

Conference2023 International Joint Conference on Neural Networks, IJCNN 2023
Country/TerritoryAustralia
CityGold Coast
Period6/18/236/23/23

Keywords

  • Capsule Network
  • Deep Learning
  • FPGA
  • Hardware Accelerator
  • Neural Network Pruning

ASJC Scopus subject areas

  • Software
  • Artificial Intelligence

Fingerprint

Dive into the research topics of 'FastCaps: A Design Methodology for Accelerating Capsule Network on Field Programmable Gate Arrays'. Together they form a unique fingerprint.

Cite this