Machine learning at facebook: Understanding inference at the edge

Carole Jean Wu; David Brooks; Kevin Chen; Douglas Chen; Sy Choudhury; Marat Dukhan; Kim Hazelwood; Eldad Isaac; Yangqing Jia; Bill Jia; Tommer Leyvand; Hao Lu; Yang Lu; Lin Qiao; Brandon Reagen; Joe Spisak; Fei Sun; Andrew Tulloch; Peter Vajda; Xiaodong Wang; Yanghan Wang; Bram Wasti; Yiming Wu; Ran Xian; Sungjoo Yoo; Peizhao Zhang

doi:10.1109/HPCA.2019.00048

Machine learning at facebook: Understanding inference at the edge

Carole Jean Wu, David Brooks, Kevin Chen, Douglas Chen, Sy Choudhury, Marat Dukhan, Kim Hazelwood, Eldad Isaac, Yangqing Jia, Bill Jia, Tommer Leyvand, Hao Lu, Yang Lu, Lin Qiao, Brandon Reagen, Joe Spisak, Fei Sun, Andrew Tulloch, Peter Vajda, Xiaodong WangYanghan Wang, Bram Wasti, Yiming Wu, Ran Xian, Sungjoo Yoo, Peizhao Zhang

Electrical and Computer Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

At Facebook, machine learning provides a wide range of capabilities that drive many aspects of user experience including ranking posts, content understanding, object detection and tracking for augmented and virtual reality, speech and text translations. While machine learning models are currently trained on customized datacenter infrastructure, Facebook is working to bring machine learning inference to the edge. By doing so, user experience is improved with reduced latency (inference time) and becomes less dependent on network connectivity. Furthermore, this also enables many more applications of deep learning with important features only made available at the edge. This paper takes a datadriven approach to present the opportunities and design challenges faced by Facebook in order to enable machine learning inference locally on smartphones and other edge platforms.

Original language	English (US)
Title of host publication	Proceedings - 25th IEEE International Symposium on High Performance Computer Architecture, HPCA 2019
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	331-344
Number of pages	14
ISBN (Electronic)	9781728114446
DOIs	https://doi.org/10.1109/HPCA.2019.00048
State	Published - Mar 26 2019
Event	25th IEEE International Symposium on High Performance Computer Architecture, HPCA 2019 - Washington, United States Duration: Feb 16 2019 → Feb 20 2019

Publication series

Name	Proceedings - 25th IEEE International Symposium on High Performance Computer Architecture, HPCA 2019

Conference

Conference	25th IEEE International Symposium on High Performance Computer Architecture, HPCA 2019
Country/Territory	United States
City	Washington
Period	2/16/19 → 2/20/19

Keywords

Edge Inference
Machine learning

ASJC Scopus subject areas

Software
Hardware and Architecture
Computer Networks and Communications

Access to Document

10.1109/HPCA.2019.00048

Cite this

Wu, C. J., Brooks, D., Chen, K., Chen, D., Choudhury, S., Dukhan, M., Hazelwood, K., Isaac, E., Jia, Y., Jia, B., Leyvand, T., Lu, H., Lu, Y., Qiao, L., Reagen, B., Spisak, J., Sun, F., Tulloch, A., Vajda, P., ... Zhang, P. (2019). Machine learning at facebook: Understanding inference at the edge. In Proceedings - 25th IEEE International Symposium on High Performance Computer Architecture, HPCA 2019 (pp. 331-344). Article 8675201 (Proceedings - 25th IEEE International Symposium on High Performance Computer Architecture, HPCA 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/HPCA.2019.00048

Machine learning at facebook: Understanding inference at the edge. / Wu, Carole Jean; Brooks, David; Chen, Kevin et al.
Proceedings - 25th IEEE International Symposium on High Performance Computer Architecture, HPCA 2019. Institute of Electrical and Electronics Engineers Inc., 2019. p. 331-344 8675201 (Proceedings - 25th IEEE International Symposium on High Performance Computer Architecture, HPCA 2019).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Wu, CJ, Brooks, D, Chen, K, Chen, D, Choudhury, S, Dukhan, M, Hazelwood, K, Isaac, E, Jia, Y, Jia, B, Leyvand, T, Lu, H, Lu, Y, Qiao, L, Reagen, B, Spisak, J, Sun, F, Tulloch, A, Vajda, P, Wang, X, Wang, Y, Wasti, B, Wu, Y, Xian, R, Yoo, S & Zhang, P 2019, Machine learning at facebook: Understanding inference at the edge. in Proceedings - 25th IEEE International Symposium on High Performance Computer Architecture, HPCA 2019., 8675201, Proceedings - 25th IEEE International Symposium on High Performance Computer Architecture, HPCA 2019, Institute of Electrical and Electronics Engineers Inc., pp. 331-344, 25th IEEE International Symposium on High Performance Computer Architecture, HPCA 2019, Washington, United States, 2/16/19. https://doi.org/10.1109/HPCA.2019.00048

Wu CJ, Brooks D, Chen K, Chen D, Choudhury S, Dukhan M et al. Machine learning at facebook: Understanding inference at the edge. In Proceedings - 25th IEEE International Symposium on High Performance Computer Architecture, HPCA 2019. Institute of Electrical and Electronics Engineers Inc. 2019. p. 331-344. 8675201. (Proceedings - 25th IEEE International Symposium on High Performance Computer Architecture, HPCA 2019). doi: 10.1109/HPCA.2019.00048

Wu, Carole Jean ; Brooks, David ; Chen, Kevin et al. / Machine learning at facebook : Understanding inference at the edge. Proceedings - 25th IEEE International Symposium on High Performance Computer Architecture, HPCA 2019. Institute of Electrical and Electronics Engineers Inc., 2019. pp. 331-344 (Proceedings - 25th IEEE International Symposium on High Performance Computer Architecture, HPCA 2019).

@inproceedings{5aa36018de9d42af892d35317ecf309b,

title = "Machine learning at facebook: Understanding inference at the edge",

abstract = "At Facebook, machine learning provides a wide range of capabilities that drive many aspects of user experience including ranking posts, content understanding, object detection and tracking for augmented and virtual reality, speech and text translations. While machine learning models are currently trained on customized datacenter infrastructure, Facebook is working to bring machine learning inference to the edge. By doing so, user experience is improved with reduced latency (inference time) and becomes less dependent on network connectivity. Furthermore, this also enables many more applications of deep learning with important features only made available at the edge. This paper takes a datadriven approach to present the opportunities and design challenges faced by Facebook in order to enable machine learning inference locally on smartphones and other edge platforms.",

keywords = "Edge Inference, Machine learning",

author = "Wu, {Carole Jean} and David Brooks and Kevin Chen and Douglas Chen and Sy Choudhury and Marat Dukhan and Kim Hazelwood and Eldad Isaac and Yangqing Jia and Bill Jia and Tommer Leyvand and Hao Lu and Yang Lu and Lin Qiao and Brandon Reagen and Joe Spisak and Fei Sun and Andrew Tulloch and Peter Vajda and Xiaodong Wang and Yanghan Wang and Bram Wasti and Yiming Wu and Ran Xian and Sungjoo Yoo and Peizhao Zhang",

note = "Publisher Copyright: {\textcopyright} 2019 IEEE.; 25th IEEE International Symposium on High Performance Computer Architecture, HPCA 2019 ; Conference date: 16-02-2019 Through 20-02-2019",

year = "2019",

month = mar,

day = "26",

doi = "10.1109/HPCA.2019.00048",

language = "English (US)",

series = "Proceedings - 25th IEEE International Symposium on High Performance Computer Architecture, HPCA 2019",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "331--344",

booktitle = "Proceedings - 25th IEEE International Symposium on High Performance Computer Architecture, HPCA 2019",

}

TY - GEN

T1 - Machine learning at facebook

T2 - 25th IEEE International Symposium on High Performance Computer Architecture, HPCA 2019

AU - Wu, Carole Jean

AU - Brooks, David

AU - Chen, Kevin

AU - Chen, Douglas

AU - Choudhury, Sy

AU - Dukhan, Marat

AU - Hazelwood, Kim

AU - Isaac, Eldad

AU - Jia, Yangqing

AU - Jia, Bill

AU - Leyvand, Tommer

AU - Lu, Hao

AU - Lu, Yang

AU - Qiao, Lin

AU - Reagen, Brandon

AU - Spisak, Joe

AU - Sun, Fei

AU - Tulloch, Andrew

AU - Vajda, Peter

AU - Wang, Xiaodong

AU - Wang, Yanghan

AU - Wasti, Bram

AU - Wu, Yiming

AU - Xian, Ran

AU - Yoo, Sungjoo

AU - Zhang, Peizhao

PY - 2019/3/26

Y1 - 2019/3/26

N2 - At Facebook, machine learning provides a wide range of capabilities that drive many aspects of user experience including ranking posts, content understanding, object detection and tracking for augmented and virtual reality, speech and text translations. While machine learning models are currently trained on customized datacenter infrastructure, Facebook is working to bring machine learning inference to the edge. By doing so, user experience is improved with reduced latency (inference time) and becomes less dependent on network connectivity. Furthermore, this also enables many more applications of deep learning with important features only made available at the edge. This paper takes a datadriven approach to present the opportunities and design challenges faced by Facebook in order to enable machine learning inference locally on smartphones and other edge platforms.

AB - At Facebook, machine learning provides a wide range of capabilities that drive many aspects of user experience including ranking posts, content understanding, object detection and tracking for augmented and virtual reality, speech and text translations. While machine learning models are currently trained on customized datacenter infrastructure, Facebook is working to bring machine learning inference to the edge. By doing so, user experience is improved with reduced latency (inference time) and becomes less dependent on network connectivity. Furthermore, this also enables many more applications of deep learning with important features only made available at the edge. This paper takes a datadriven approach to present the opportunities and design challenges faced by Facebook in order to enable machine learning inference locally on smartphones and other edge platforms.

KW - Edge Inference

KW - Machine learning

UR - http://www.scopus.com/inward/record.url?scp=85064189318&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85064189318&partnerID=8YFLogxK

U2 - 10.1109/HPCA.2019.00048

DO - 10.1109/HPCA.2019.00048

M3 - Conference contribution

AN - SCOPUS:85064189318

T3 - Proceedings - 25th IEEE International Symposium on High Performance Computer Architecture, HPCA 2019

SP - 331

EP - 344

BT - Proceedings - 25th IEEE International Symposium on High Performance Computer Architecture, HPCA 2019

PB - Institute of Electrical and Electronics Engineers Inc.

Y2 - 16 February 2019 through 20 February 2019

ER -

Machine learning at facebook: Understanding inference at the edge

Abstract

Publication series

Conference

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this