TY - GEN
T1 - Very Long Term Field of View Prediction for 360-Degree Video Streaming
AU - Li, Chenge
AU - Zhang, Weixi
AU - Liu, Yong
AU - Wang, Yao
N1 - Publisher Copyright:
© 2019 IEEE.
Copyright:
Copyright 2019 Elsevier B.V., All rights reserved.
PY - 2019/4/22
Y1 - 2019/4/22
N2 - 360-degree videos have gained increasing popularity in recent years with the developments and advances in Virtual Reality (VR) and Augmented Reality (AR) technologies. In such applications, a user only watches a video scene within a field of view (FoV) centered in a certain direction. Predicting the future FoV in a long time horizon (more than seconds ahead) can help save bandwidth resources in on-demand video streaming while minimizing video freezing in networks with significant bandwidth variations. In this work, we treat the FoV prediction as a sequence learning problem, and propose to predict the target user's future FoV not only based on the user's own past FoV center trajectory but also other users' future FoV locations. We propose multiple prediction models based on two different FoV representations: One using FoV center trajectories and another using equirectangular heatmaps that represent the FoV center distributions. Extensive evaluations with two public datasets demonstrate that the proposed models can significantly outperform benchmark models, and other users' FoVs are very helpful for improving long-term predictions.
AB - 360-degree videos have gained increasing popularity in recent years with the developments and advances in Virtual Reality (VR) and Augmented Reality (AR) technologies. In such applications, a user only watches a video scene within a field of view (FoV) centered in a certain direction. Predicting the future FoV in a long time horizon (more than seconds ahead) can help save bandwidth resources in on-demand video streaming while minimizing video freezing in networks with significant bandwidth variations. In this work, we treat the FoV prediction as a sequence learning problem, and propose to predict the target user's future FoV not only based on the user's own past FoV center trajectory but also other users' future FoV locations. We propose multiple prediction models based on two different FoV representations: One using FoV center trajectories and another using equirectangular heatmaps that represent the FoV center distributions. Extensive evaluations with two public datasets demonstrate that the proposed models can significantly outperform benchmark models, and other users' FoVs are very helpful for improving long-term predictions.
KW - 360-degree video streaming
KW - field of view
KW - time series prediction
KW - virtual reality
UR - http://www.scopus.com/inward/record.url?scp=85065627631&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85065627631&partnerID=8YFLogxK
U2 - 10.1109/MIPR.2019.00060
DO - 10.1109/MIPR.2019.00060
M3 - Conference contribution
AN - SCOPUS:85065627631
T3 - Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019
SP - 297
EP - 302
BT - Proceedings - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 2nd International Conference on Multimedia Information Processing and Retrieval, MIPR 2019
Y2 - 28 March 2019 through 30 March 2019
ER -