Low-latency FoV-adaptive Coding and Streaming for Interactive 360° Video Streaming

Yixiang Mao, Liyang Sun, Yong Liu, Yao Wang

Research output: Chapter in Book/Report/Conference proceedingConference contribution


In 360° video interactive streaming, it is critical to minimize the end-to-end frame delay. It is also important to predict the user's field of video (FoV) and allocate more bits in regions within the predicted FoV. Towards both goals, we propose a low-delay FoVadaptive coding and delivery system that is robust to bandwidth variations and FoV prediction errors. Each frame is coded only in the predicted FoV (PF), a border surrounding the predicted FoV (PF+), and a rotating intra (RI) region. To maximize the coding efficiency, the PF and PF+ regions are coded with temporal and spatial prediction, while the RI region is coded with spatial prediction only. The RI region enables periodic refreshment of the entire frame and provides robustness to both FoV prediction errors and frame losses. The total bit budget is adapted both at the segment level based on the predicted average bandwidth for the segment and at the frame level based on the sender buffer status, to ensure timely delivery. The system further adapts the sizes and coding rates of different regions for each video segment to maximize the average rendered video quality under the total bit budget. To enable such adaptation, we propose novel ways to model the quality-rate (Q-R) relations of coded regions that take into account of potentially misaligned coded regions in successive frames due to FoV dynamics. We examine the performance of the proposed system and three benchmark systems, under real-world bandwidth traces and FoV traces, and demonstrate that the proposed system can significantly improve the rendered video quality over the benchmark systems. Furthermore, the proposed system can achieve very low end-to-end frame delay while maintaining a low frame freeze probability and providing smooth video playback.

Original languageEnglish (US)
Title of host publicationMM 2020 - Proceedings of the 28th ACM International Conference on Multimedia
PublisherAssociation for Computing Machinery, Inc
Number of pages9
ISBN (Electronic)9781450379885
StatePublished - Oct 12 2020
Event28th ACM International Conference on Multimedia, MM 2020 - Virtual, Online, United States
Duration: Oct 12 2020Oct 16 2020

Publication series

NameMM 2020 - Proceedings of the 28th ACM International Conference on Multimedia


Conference28th ACM International Conference on Multimedia, MM 2020
Country/TerritoryUnited States
CityVirtual, Online


  • 360-degree video
  • fov-adaptive streaming
  • low latency
  • tile-based coding

ASJC Scopus subject areas

  • Software
  • Computer Graphics and Computer-Aided Design
  • Human-Computer Interaction


Dive into the research topics of 'Low-latency FoV-adaptive Coding and Streaming for Interactive 360° Video Streaming'. Together they form a unique fingerprint.

Cite this