Coupling parameter and particle dynamics for adaptive sampling in Neural Galerkin schemes

Yuxiao Wen, Eric Vanden-Eijnden, Benjamin Peherstorfer

Research output: Contribution to journalArticlepeer-review

Abstract

Training nonlinear parametrizations such as deep neural networks to numerically approximate solutions of partial differential equations is often based on minimizing a loss that includes the residual, which is analytically available in limited settings only. At the same time, empirically estimating the training loss is challenging because residuals and related quantities can have high variance, especially for transport-dominated and high-dimensional problems that exhibit local features such as waves and coherent structures. Thus, estimators based on data samples from un-informed, uniform distributions are inefficient. This work introduces Neural Galerkin schemes that estimate the training loss with data from adaptive distributions, which are empirically represented via ensembles of particles. The ensembles are actively adapted by evolving the particles with dynamics coupled to the nonlinear parametrizations of the solution fields so that the ensembles remain informative for estimating the training loss. Numerical experiments indicate that few dynamic particles are sufficient for obtaining accurate empirical estimates of the training loss, even for problems with local features and with high-dimensional spatial domains.

Original languageEnglish (US)
Article number134129
JournalPhysica D: Nonlinear Phenomena
Volume462
DOIs
StatePublished - Jun 2024

Keywords

  • Active learning
  • Deep neural networks
  • Model reduction
  • Numerical analysis
  • Scientific machine learning

ASJC Scopus subject areas

  • Statistical and Nonlinear Physics
  • Mathematical Physics
  • Condensed Matter Physics
  • Applied Mathematics

Fingerprint

Dive into the research topics of 'Coupling parameter and particle dynamics for adaptive sampling in Neural Galerkin schemes'. Together they form a unique fingerprint.

Cite this