TY - GEN
T1 - Deep Multi Depth Panoramas for View Synthesis
AU - Lin, Kai En
AU - Xu, Zexiang
AU - Mildenhall, Ben
AU - Srinivasan, Pratul P.
AU - Hold-Geoffroy, Yannick
AU - DiVerdi, Stephen
AU - Sun, Qi
AU - Sunkavalli, Kalyan
AU - Ramamoorthi, Ravi
N1 - Publisher Copyright:
© 2020, Springer Nature Switzerland AG.
PY - 2020
Y1 - 2020
N2 - We propose a learning-based approach for novel view synthesis for multi-camera 360∘ panorama capture rigs. Previous work constructs RGBD panoramas from such data, allowing for view synthesis with small amounts of translation, but cannot handle the disocclusions and view-dependent effects that are caused by large translations. To address this issue, we present a novel scene representation—Multi Depth Panorama (MDP)—that consists of multiple RGBDα panoramas that represent both scene geometry and appearance. We demonstrate a deep neural network-based method to reconstruct MDPs from multi-camera 360∘ images. MDPs are more compact than previous 3D scene representations and enable high-quality, efficient new view rendering. We demonstrate this via experiments on both synthetic and real data and comparisons with previous state-of-the-art methods spanning both learning-based approaches and classical RGBD-based methods.
AB - We propose a learning-based approach for novel view synthesis for multi-camera 360∘ panorama capture rigs. Previous work constructs RGBD panoramas from such data, allowing for view synthesis with small amounts of translation, but cannot handle the disocclusions and view-dependent effects that are caused by large translations. To address this issue, we present a novel scene representation—Multi Depth Panorama (MDP)—that consists of multiple RGBDα panoramas that represent both scene geometry and appearance. We demonstrate a deep neural network-based method to reconstruct MDPs from multi-camera 360∘ images. MDPs are more compact than previous 3D scene representations and enable high-quality, efficient new view rendering. We demonstrate this via experiments on both synthetic and real data and comparisons with previous state-of-the-art methods spanning both learning-based approaches and classical RGBD-based methods.
KW - 360 panoramas
KW - Image-based rendering
KW - View synthesis
KW - Virtual reality
UR - http://www.scopus.com/inward/record.url?scp=85097621766&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85097621766&partnerID=8YFLogxK
U2 - 10.1007/978-3-030-58601-0_20
DO - 10.1007/978-3-030-58601-0_20
M3 - Conference contribution
AN - SCOPUS:85097621766
SN - 9783030586003
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 328
EP - 344
BT - Computer Vision – ECCV 2020 - 16th European Conference, 2020, Proceedings
A2 - Vedaldi, Andrea
A2 - Bischof, Horst
A2 - Brox, Thomas
A2 - Frahm, Jan-Michael
PB - Springer Science and Business Media Deutschland GmbH
T2 - 16th European Conference on Computer Vision, ECCV 2020
Y2 - 23 August 2020 through 28 August 2020
ER -