Convolutional neural network-based coefficients prediction for hevc intra-predicted residues

Changyue Ma, Dong Liu, Li Li, Yao Wang, Feng Wu

Research output: Chapter in Book/Report/Conference proceedingConference contribution


We propose a convolutional neural network-based coefficients prediction (CNNCP) method for intra-predicted residues in the High Efficiency Video Coding (HEVC) standard. In HEVC, discrete cosine transform (DCT) or discrete sine transform (DST) is adopted to convert the intra-predicted residues in the spatial domain into coefficients in the frequency domain. Each coefficient is scalar quantized and entropy coded into the bitstream. As DCT or DST is non-optimal linear transform, there still exist linear and non-linear correlations among different coefficients after the transform. In addition, there exist coefficients' correlations between current block and neighboring blocks, as these correlations cannot be completely exploited in the intra prediction. We thus propose to perform coefficients prediction to further reduce the redundancy among coefficients. The coefficients prediction is achieved using trained convolutional neural networks (CNNs), as CNNs can build complex relationship between input and output by training with a lot of data. In addition, a flag that signals whether to perform coefficients prediction or not at the coding unit level is transmitted to decoder. The proposed CNNCP method is implemented upon the HEVC reference software. Experimental results show that the proposed method achieves on average 1.8%, 4.1%, and 4.5% BD-rate reduction ratios in Y, U, V, respectively, compared with the HEVC baseline in all-intra configuration. In particular, the average BD-rate reduction ratios for 4K test sequences are 2.9%, 6.5%, and 6.6%.

Original languageEnglish (US)
Title of host publicationProceedings - DCC 2020
Subtitle of host publicationData Compression Conference
EditorsAli Bilgin, Michael W. Marcellin, Joan Serra-Sagrista, James A. Storer
PublisherInstitute of Electrical and Electronics Engineers Inc.
Number of pages10
ISBN (Electronic)9781728164571
StatePublished - Mar 2020
Event2020 Data Compression Conference, DCC 2020 - Snowbird, United States
Duration: Mar 24 2020Mar 27 2020

Publication series

NameData Compression Conference Proceedings
ISSN (Print)1068-0314


Conference2020 Data Compression Conference, DCC 2020
Country/TerritoryUnited States

ASJC Scopus subject areas

  • Computer Networks and Communications


Dive into the research topics of 'Convolutional neural network-based coefficients prediction for hevc intra-predicted residues'. Together they form a unique fingerprint.

Cite this