A survey on GAN acceleration using memory compression techniques

Dina Tantawy, Mohamed Zahran, Amr Wassal

Research output: Contribution to journalReview articlepeer-review


Since its invention, generative adversarial networks (GANs) have shown outstanding results in many applications. GANs are powerful, yet resource-hungry deep learning models. The main difference between GANs and ordinary deep learning models is the nature of their output and training instability. For example, GANs output can be a whole image versus other models detecting objects or classifying images. Thus, the architecture and numeric precision of the network affect the quality and speed of the solution. Hence, accelerating GANs is pivotal. Data transfer is considered the main source of energy consumption, that is why memory compression is a very efficient technique to accelerate and optimize GANs. Two main types of memory compression exist: lossless and lossy ones. Lossless compression techniques are general among all models; thus, we will focus in this paper on lossy techniques. Lossy compression techniques are further classified into (a) pruning, (b) knowledge distillation, (c) low-rank factorization, (d) lowering numeric precision, and (e) encoding. In this paper, we survey lossy compression techniques for CNN-based GANs. Our findings showed the superiority of knowledge distillation over pruning alone and the gaps in the research field that needs to be explored like encoding and different combination of compression techniques.

Original languageEnglish (US)
Article number47
JournalJournal of Engineering and Applied Science
Issue number1
StatePublished - Dec 2021


  • Acceleration
  • Compression
  • Generative adversarial networks
  • Knowledge distillation
  • Optimization
  • Pruning
  • Quantization
  • Survey

ASJC Scopus subject areas

  • General Engineering


Dive into the research topics of 'A survey on GAN acceleration using memory compression techniques'. Together they form a unique fingerprint.

Cite this