A framework for low communication approaches for large scale 3D convolution

Anuva Kulkarni, Jelena Kovacevic, Franz Franchetti

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Large-scale 3D convolutions computed using parallel Fast Fourier Transforms (FFTs) demand multiple all-To-All communication steps, which cause bottlenecks on computing clusters. Since data transfer speeds to/from memory have not increased proportionally to computational capacity (in terms of FLOPs), 3D FFTs become bounded by communication and are difficult to scale, especially on modern heterogeneous computing platforms consisting of accelerators like GPUs. Existing HPC frameworks focus on optimizing the isolated FFT algorithm or communication patterns, but still require multiple all-To-All communication steps during convolution. In this work, we present a strategy for scalable convolution such that it avoids multiple all-To-All exchanges, and also optimizes necessary communication. We provide proof-of-concept results under assumptions of a use case, the MASSIF Hooke's law simulation convolution kernel. Our method localizes computation by exploiting properties of the data, and approximates the convolution result by data compression, resulting in increased scalability of 3D convolution. Our preliminary results show scalability of 8 times more than traditional methods in the same compute resources without adversely affecting result accuracy. Our method can be adapted for first-principle scientific simulations and leverages cross-disciplinary knowledge of the application, the data and computing to perform large-scale convolution while avoiding communication bottlenecks. In order to make our approach widely usable and adaptable for emerging challenges, we discuss the use of FFTX, a novel framework which can be used for platform-Agnostic specification and optimization for algorithmic approaches similar to ours.

Original languageEnglish (US)
Title of host publication51st International Conference on Parallel Processing, ICPP 2022 - Workshop Proceedings
PublisherAssociation for Computing Machinery
ISBN (Electronic)9781450394451
StatePublished - Aug 29 2022
Event51st International Conference on Parallel Processing, ICPP 2022 - Virtual, Online, France
Duration: Aug 29 2022Sep 1 2022

Publication series

NameACM International Conference Proceeding Series


Conference51st International Conference on Parallel Processing, ICPP 2022
CityVirtual, Online


  • Fast Fourier Transform
  • GPU
  • Green's functions
  • Scalable Convolutions
  • scientific simulations

ASJC Scopus subject areas

  • Software
  • Human-Computer Interaction
  • Computer Vision and Pattern Recognition
  • Computer Networks and Communications


Dive into the research topics of 'A framework for low communication approaches for large scale 3D convolution'. Together they form a unique fingerprint.

Cite this