GAN Training With Kernel Discriminators: What Parameters Control Convergence Rates?

Evan Becker, Parthe Pandit, Sundeep Rangan, Alyson K. Fletcher

Research output: Contribution to journalArticlepeer-review

Abstract

—Generative Adversarial Networks (GANs) are widely used for modeling complex data. However, the dynamics of the gradient descent-ascent (GDA) algorithms, often used for training GANs, have been notoriously difficult to analyze. We study these dynamics in the case where the discriminator is kernel-based and the true distribution consists of discrete points in Euclidean space. Prior works have analyzed the GAN dynamics in such scenarios via simple linearization close to the equilibrium. In this work, we show that linearized analysis can be grossly inaccurate, even at moderate distances from the equilibrium. We then propose an alternative non-linear yet tractable second moment model. The proposed model predicts the convergence behavior well and reveals new insights about the role of the kernel width on convergence rate, not apparent in the linearized analysis. These insights suggest certain shapes of the kernel offer both fast local convergence and improved global convergence. We corroborate our theoretical results through simulations.

Original languageEnglish (US)
Pages (from-to)433-445
Number of pages13
JournalIEEE Transactions on Signal Processing
Volume73
DOIs
StatePublished - 2025

Keywords

  • Convergence rates
  • GANs
  • kernel machines
  • stability

ASJC Scopus subject areas

  • Signal Processing
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'GAN Training With Kernel Discriminators: What Parameters Control Convergence Rates?'. Together they form a unique fingerprint.

Cite this