Expectation consistent approximate inference: Generalizations and convergence

Alyson Fletcher; Mojtaba Sahraee-Ardakan; Sundeep Rangan; Philip Schniter

doi:10.1109/ISIT.2016.7541287

Expectation consistent approximate inference: Generalizations and convergence

Alyson Fletcher, Mojtaba Sahraee-Ardakan, Sundeep Rangan, Philip Schniter

Electrical and Computer Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Approximations of loopy belief propagation, including expectation propagation and approximate message passing, have attracted considerable attention for probabilistic inference problems. This paper proposes and analyzes a generalization of Opper and Winther's expectation consistent (EC) approximate inference method. The proposed method, called Generalized Expectation Consistency (GEC), can be applied to both maximum a posteriori (MAP) and minimum mean squared error (MMSE) estimation. Here we characterize its fixed points, convergence, and performance relative to the replica prediction of optimality.

Original language	English (US)
Title of host publication	Proceedings - ISIT 2016; 2016 IEEE International Symposium on Information Theory
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	190-194
Number of pages	5
ISBN (Electronic)	9781509018062
DOIs	https://doi.org/10.1109/ISIT.2016.7541287
State	Published - Aug 10 2016
Event	2016 IEEE International Symposium on Information Theory, ISIT 2016 - Barcelona, Spain Duration: Jul 10 2016 → Jul 15 2016

Publication series

Name	IEEE International Symposium on Information Theory - Proceedings
Volume	2016-August
ISSN (Print)	2157-8095

Other

Other	2016 IEEE International Symposium on Information Theory, ISIT 2016
Country/Territory	Spain
City	Barcelona
Period	7/10/16 → 7/15/16

Keywords

Approximate message passing
Bethe free energy
Expectation propagation
S-transform in free probability

ASJC Scopus subject areas

Theoretical Computer Science
Information Systems
Modeling and Simulation
Applied Mathematics

Access to Document

10.1109/ISIT.2016.7541287

Cite this

Fletcher, A., Sahraee-Ardakan, M., Rangan, S., & Schniter, P. (2016). Expectation consistent approximate inference: Generalizations and convergence. In Proceedings - ISIT 2016; 2016 IEEE International Symposium on Information Theory (pp. 190-194). Article 7541287 (IEEE International Symposium on Information Theory - Proceedings; Vol. 2016-August). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ISIT.2016.7541287

Expectation consistent approximate inference: Generalizations and convergence. / Fletcher, Alyson; Sahraee-Ardakan, Mojtaba; Rangan, Sundeep et al.
Proceedings - ISIT 2016; 2016 IEEE International Symposium on Information Theory. Institute of Electrical and Electronics Engineers Inc., 2016. p. 190-194 7541287 (IEEE International Symposium on Information Theory - Proceedings; Vol. 2016-August).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Fletcher, A, Sahraee-Ardakan, M, Rangan, S & Schniter, P 2016, Expectation consistent approximate inference: Generalizations and convergence. in Proceedings - ISIT 2016; 2016 IEEE International Symposium on Information Theory., 7541287, IEEE International Symposium on Information Theory - Proceedings, vol. 2016-August, Institute of Electrical and Electronics Engineers Inc., pp. 190-194, 2016 IEEE International Symposium on Information Theory, ISIT 2016, Barcelona, Spain, 7/10/16. https://doi.org/10.1109/ISIT.2016.7541287

Fletcher A, Sahraee-Ardakan M, Rangan S, Schniter P. Expectation consistent approximate inference: Generalizations and convergence. In Proceedings - ISIT 2016; 2016 IEEE International Symposium on Information Theory. Institute of Electrical and Electronics Engineers Inc. 2016. p. 190-194. 7541287. (IEEE International Symposium on Information Theory - Proceedings). doi: 10.1109/ISIT.2016.7541287

Fletcher, Alyson ; Sahraee-Ardakan, Mojtaba ; Rangan, Sundeep et al. / Expectation consistent approximate inference : Generalizations and convergence. Proceedings - ISIT 2016; 2016 IEEE International Symposium on Information Theory. Institute of Electrical and Electronics Engineers Inc., 2016. pp. 190-194 (IEEE International Symposium on Information Theory - Proceedings).

@inproceedings{d0a19fd3dfe448bea5eb9ee4ed503c32,

title = "Expectation consistent approximate inference: Generalizations and convergence",

abstract = "Approximations of loopy belief propagation, including expectation propagation and approximate message passing, have attracted considerable attention for probabilistic inference problems. This paper proposes and analyzes a generalization of Opper and Winther's expectation consistent (EC) approximate inference method. The proposed method, called Generalized Expectation Consistency (GEC), can be applied to both maximum a posteriori (MAP) and minimum mean squared error (MMSE) estimation. Here we characterize its fixed points, convergence, and performance relative to the replica prediction of optimality.",

keywords = "Approximate message passing, Bethe free energy, Expectation propagation, S-transform in free probability",

author = "Alyson Fletcher and Mojtaba Sahraee-Ardakan and Sundeep Rangan and Philip Schniter",

note = "Publisher Copyright: {\textcopyright} 2016 IEEE.; 2016 IEEE International Symposium on Information Theory, ISIT 2016 ; Conference date: 10-07-2016 Through 15-07-2016",

year = "2016",

month = aug,

day = "10",

doi = "10.1109/ISIT.2016.7541287",

language = "English (US)",

series = "IEEE International Symposium on Information Theory - Proceedings",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "190--194",

booktitle = "Proceedings - ISIT 2016; 2016 IEEE International Symposium on Information Theory",

}

TY - GEN

T1 - Expectation consistent approximate inference

T2 - 2016 IEEE International Symposium on Information Theory, ISIT 2016

AU - Fletcher, Alyson

AU - Sahraee-Ardakan, Mojtaba

AU - Rangan, Sundeep

AU - Schniter, Philip

PY - 2016/8/10

Y1 - 2016/8/10

N2 - Approximations of loopy belief propagation, including expectation propagation and approximate message passing, have attracted considerable attention for probabilistic inference problems. This paper proposes and analyzes a generalization of Opper and Winther's expectation consistent (EC) approximate inference method. The proposed method, called Generalized Expectation Consistency (GEC), can be applied to both maximum a posteriori (MAP) and minimum mean squared error (MMSE) estimation. Here we characterize its fixed points, convergence, and performance relative to the replica prediction of optimality.

AB - Approximations of loopy belief propagation, including expectation propagation and approximate message passing, have attracted considerable attention for probabilistic inference problems. This paper proposes and analyzes a generalization of Opper and Winther's expectation consistent (EC) approximate inference method. The proposed method, called Generalized Expectation Consistency (GEC), can be applied to both maximum a posteriori (MAP) and minimum mean squared error (MMSE) estimation. Here we characterize its fixed points, convergence, and performance relative to the replica prediction of optimality.

KW - Approximate message passing

KW - Bethe free energy

KW - Expectation propagation

KW - S-transform in free probability

UR - http://www.scopus.com/inward/record.url?scp=84985987005&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84985987005&partnerID=8YFLogxK

U2 - 10.1109/ISIT.2016.7541287

DO - 10.1109/ISIT.2016.7541287

M3 - Conference contribution

AN - SCOPUS:84985987005

T3 - IEEE International Symposium on Information Theory - Proceedings

SP - 190

EP - 194

BT - Proceedings - ISIT 2016; 2016 IEEE International Symposium on Information Theory

PB - Institute of Electrical and Electronics Engineers Inc.

Y2 - 10 July 2016 through 15 July 2016

ER -

Expectation consistent approximate inference: Generalizations and convergence

Abstract

Publication series

Other

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this