TY - GEN
T1 - Deep Multi Depth Panoramas for View Synthesis
AU - Lin, Kai En
AU - Xu, Zexiang
AU - Mildenhall, Ben
AU - Srinivasan, Pratul P.
AU - Hold-Geoffroy, Yannick
AU - DiVerdi, Stephen
AU - Sun, Qi
AU - Sunkavalli, Kalyan
AU - Ramamoorthi, Ravi
N1 - Funding Information:
Acknowledgements. We would like to thank In-Kyu Park for helpful discussion and comments. This work was supported in part by ONR grants N000141712687, N000141912293, N000142012529, NSF grant 1617234, Adobe, the Ronald L. Graham Chair and the UC San Diego Center for Visual Computing.
Funding Information:
We would like to thank In-Kyu Park for helpful discussion and comments. This work was supported in part by ONR grants N000141712687, N000141912293, N000142012529, NSF grant 1617234, Adobe, the Ronald L. Graham Chair and the UC San Diego Center for Visual Computing.
Publisher Copyright:
© 2020, Springer Nature Switzerland AG.
PY - 2020
Y1 - 2020
N2 - We propose a learning-based approach for novel view synthesis for multi-camera 360∘ panorama capture rigs. Previous work constructs RGBD panoramas from such data, allowing for view synthesis with small amounts of translation, but cannot handle the disocclusions and view-dependent effects that are caused by large translations. To address this issue, we present a novel scene representation—Multi Depth Panorama (MDP)—that consists of multiple RGBDα panoramas that represent both scene geometry and appearance. We demonstrate a deep neural network-based method to reconstruct MDPs from multi-camera 360∘ images. MDPs are more compact than previous 3D scene representations and enable high-quality, efficient new view rendering. We demonstrate this via experiments on both synthetic and real data and comparisons with previous state-of-the-art methods spanning both learning-based approaches and classical RGBD-based methods.
AB - We propose a learning-based approach for novel view synthesis for multi-camera 360∘ panorama capture rigs. Previous work constructs RGBD panoramas from such data, allowing for view synthesis with small amounts of translation, but cannot handle the disocclusions and view-dependent effects that are caused by large translations. To address this issue, we present a novel scene representation—Multi Depth Panorama (MDP)—that consists of multiple RGBDα panoramas that represent both scene geometry and appearance. We demonstrate a deep neural network-based method to reconstruct MDPs from multi-camera 360∘ images. MDPs are more compact than previous 3D scene representations and enable high-quality, efficient new view rendering. We demonstrate this via experiments on both synthetic and real data and comparisons with previous state-of-the-art methods spanning both learning-based approaches and classical RGBD-based methods.
KW - 360 panoramas
KW - Image-based rendering
KW - View synthesis
KW - Virtual reality
UR - http://www.scopus.com/inward/record.url?scp=85097621766&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85097621766&partnerID=8YFLogxK
U2 - 10.1007/978-3-030-58601-0_20
DO - 10.1007/978-3-030-58601-0_20
M3 - Conference contribution
AN - SCOPUS:85097621766
SN - 9783030586003
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 328
EP - 344
BT - Computer Vision – ECCV 2020 - 16th European Conference, 2020, Proceedings
A2 - Vedaldi, Andrea
A2 - Bischof, Horst
A2 - Brox, Thomas
A2 - Frahm, Jan-Michael
PB - Springer Science and Business Media Deutschland GmbH
T2 - 16th European Conference on Computer Vision, ECCV 2020
Y2 - 23 August 2020 through 28 August 2020
ER -