Inference with deep generative priors in high dimensions

Parthe Pandit, Mojtaba Sahraee-Ardakan, Sundeep Rangan, Philip Schniter, Alyson K. Fletcher

Research output: Contribution to journalArticlepeer-review


Deep generative priors offer powerful models for complex-structured data, such as images, audio, and text. Using these priors in inverse problems typically requires estimating the input and/or hidden signals in a multi-layer deep neural network from observation of its output. While these approaches have been successful in practice, rigorous performance analysis is complicated by the non-convex nature of the underlying optimization problems. This paper presents a novel algorithm, Multi-Layer Vector Approximate Message Passing (ML-VAMP), for inference in multi-layer stochastic neural networks. ML-VAMP can be configured to compute maximum a priori (MAP) or approximate minimum mean-squared error (MMSE) estimates for these networks. We show that the performance of ML-VAMP can be exactly predicted in a certain high-dimensional random limit. Furthermore, under certain conditions, ML-VAMP yields estimates that achieve the minimum (i.e., Bayes-optimal) MSE as predicted by the replica method. In this way, ML-VAMP provides a computationally efficient method for multi-layer inference with an exact performance characterization and testable conditions for optimality in the large-system limit.

Original languageEnglish (US)
Article number2986321
Pages (from-to)336-347
Number of pages12
JournalIEEE Journal on Selected Areas in Information Theory
Issue number1
StatePublished - May 2020


  • Analyzing deep neural networks
  • Inverse problems
  • State evolution
  • Stochastic neural networks
  • Vector approximate message passing

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Media Technology
  • Artificial Intelligence
  • Applied Mathematics


Dive into the research topics of 'Inference with deep generative priors in high dimensions'. Together they form a unique fingerprint.

Cite this