Cross-Layer Optimizations for Efficient Deep Learning Inference at the Edge

The capability of directly and accurately learning pattern from raw data has enabled the use of deep learning (DL) in various application domains such as image and video processing, natural language processing, autonomous driving, robotics, smart healthcare, and predictive maintenance. However, the state-of-the-art DL models are highly complex and require a large amount of compute and memory resources to generate highly accurate results. A number of DNN optimization techniques have been proposed to reduce the complexity of these models and enable their reliable and efficient deployment in resource-constrained embedded scenarios. This chapter first presents an overview of different DNN optimization and approximation techniques designed to improve area, performance, and energy efficiency of DL systems. Then, it covers different methodologies that synergistically integrate different optimization techniques in a cross-layer flow. Toward the end, the chapter discusses works related to end-to-end system-level optimizations and approximations to enable ultra-efficient DNN inference.