TY - GEN
T1 - Asymptotics of MAP Inference in Deep Networks
AU - Pandit, Parthe
AU - Sahraee, Mojtaba
AU - Rangan, Sundeep
AU - Fletcher, Alyson K.
N1 - Funding Information:
AKF was supported by NSF grants 1254204 and 1738286, and ONR N00014-15-1-2677. SR was supported by NSF Grants 1116589, 1302336, and 1547332, NIST award 70NANB17H166, SRC, and NYU WIRELESS.
Publisher Copyright:
© 2019 IEEE.
PY - 2019/7
Y1 - 2019/7
N2 - Deep generative priors are a powerful tool for reconstruction problems with complex data such as images and text. Inverse problems using such models require solving an inference problem of estimating the input and hidden units of the multi-layer network from its output. Maximum a priori (MAP) estimation is a widely-used inference method as it is straightforward to implement, and has been successful in practice. However, rigorous analysis of MAP inference in multi-layer networks is difficult. This work considers a recently-developed method, multilayer vector approximate message passing (ML-VAMP), to study MAP inference in deep networks. It is shown that the mean squared error of the ML-VAMP estimate can be exactly and rigorously characterized in a certain high-dimensional random limit. The proposed method thus provides a tractable method for MAP inference with exact performance guarantees.
AB - Deep generative priors are a powerful tool for reconstruction problems with complex data such as images and text. Inverse problems using such models require solving an inference problem of estimating the input and hidden units of the multi-layer network from its output. Maximum a priori (MAP) estimation is a widely-used inference method as it is straightforward to implement, and has been successful in practice. However, rigorous analysis of MAP inference in multi-layer networks is difficult. This work considers a recently-developed method, multilayer vector approximate message passing (ML-VAMP), to study MAP inference in deep networks. It is shown that the mean squared error of the ML-VAMP estimate can be exactly and rigorously characterized in a certain high-dimensional random limit. The proposed method thus provides a tractable method for MAP inference with exact performance guarantees.
UR - http://www.scopus.com/inward/record.url?scp=85073171878&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85073171878&partnerID=8YFLogxK
U2 - 10.1109/ISIT.2019.8849316
DO - 10.1109/ISIT.2019.8849316
M3 - Conference contribution
AN - SCOPUS:85073171878
T3 - IEEE International Symposium on Information Theory - Proceedings
SP - 842
EP - 846
BT - 2019 IEEE International Symposium on Information Theory, ISIT 2019 - Proceedings
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 2019 IEEE International Symposium on Information Theory, ISIT 2019
Y2 - 7 July 2019 through 12 July 2019
ER -