3DensiNet: A robust neural network architecture towards 3D volumetric object prediction from 2D image

Meng Wang, Lingjing Wang, Yi Fang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

3D volumetric object generation/prediction from single 2D image is a quite challenging but meaningful task in 3D visual computing. In this paper, we propose a novel neural network architecture, named "3DensiNet", which uses density heat-map as an intermediate supervision tool for 2D-to-3D transformation. Specifically, we firstly present a 2D density heat-map to 3D volumetric object encoding-decoding network, which outperforms classical 3D autoencoder. Then we show that using 2D image to predict its density heat-map via a 2D to 2D encoding-decoding network is feasible. In addition, we leverage adversarial loss to fine tune our network, which improves the generated/predicted 3D voxel objects to be more similar to the ground truth voxel object. Experimental results on 3D volumetric prediction from 2D images demonstrates superior performance of 3DensiNet over other state-of-the-art techniques in handling 3D volumetric object generation/prediction from single 2D image.

Original languageEnglish (US)
Title of host publicationMM 2017 - Proceedings of the 2017 ACM Multimedia Conference
PublisherAssociation for Computing Machinery, Inc
Pages961-969
Number of pages9
ISBN (Electronic)9781450349062
DOIs
StatePublished - Oct 23 2017
Event25th ACM International Conference on Multimedia, MM 2017 - Mountain View, United States
Duration: Oct 23 2017Oct 27 2017

Publication series

NameMM 2017 - Proceedings of the 2017 ACM Multimedia Conference

Other

Other25th ACM International Conference on Multimedia, MM 2017
CountryUnited States
CityMountain View
Period10/23/1710/27/17

Keywords

  • 3D reconstruction
  • 3D volumetric prediction
  • Deep learning

ASJC Scopus subject areas

  • Computer Graphics and Computer-Aided Design
  • Media Technology
  • Computer Vision and Pattern Recognition
  • Software

Fingerprint Dive into the research topics of '3DensiNet: A robust neural network architecture towards 3D volumetric object prediction from 2D image'. Together they form a unique fingerprint.

  • Cite this

    Wang, M., Wang, L., & Fang, Y. (2017). 3DensiNet: A robust neural network architecture towards 3D volumetric object prediction from 2D image. In MM 2017 - Proceedings of the 2017 ACM Multimedia Conference (pp. 961-969). (MM 2017 - Proceedings of the 2017 ACM Multimedia Conference). Association for Computing Machinery, Inc. https://doi.org/10.1145/3123266.3123340