Computational multi-view imaging with kinect

Xinchen Ye, Jingyu Yang, Hao Huang, Chunping Hou, Yao Wang

Research output: Contribution to journalArticle

Abstract

The lack of 3-D content has become a bottleneck for the advancement of three-dimensional television (3-DTV), but conventional multicamera arrays for multiview imaging are expensive to setup and cumbersome to use. This paper proposes a lightweight multiview imaging approach with Kinect, a handheld integrated depth-color camera, under the depth-image-based rendering framework. The proposed method consists of two components: depth restoration from noisy and incomplete depth measurements and view synthesis from depth-color pairs. In depth restoration, we propose a moving 2-D polynomial approximation via least squares to suppress quantization errors in the acquired depth values, and propose a progressive edge-guided trilateral filter to fill missing areas of the depth map. Edges extracted from color image are used to predict the locations of depth discontinuities in missing areas and to guide the proposed trilateral filter avoiding filtering across discontinuities. In view synthesis, we propose a low-rank matrix restoration model to inpaint disocclusion regions, fully exploiting the nonlocal correlations in images, and devise an efficient algorithm under the augmented lagrange multiplier (ALM) framework. Disocclusion areas are inpainted progressively from the boundaries of disocclusion with an estimated priority consisting of four terms: warping term, reliability term, texture term, and depth term. Experimental results show that our method restores high quality depth maps even for large missing areas, and synthesizes natural multiview images from restored depth maps. Strong 3-D visual experiences are observed when the synthesized multiview images are shown in two types of stereoscopic displays.

Original languageEnglish (US)
Article number6881679
Pages (from-to)540-544
Number of pages5
JournalIEEE Transactions on Broadcasting
Volume60
Issue number3
DOIs
StatePublished - Sep 1 2014

Keywords

  • Kinect
  • Multiview imaging
  • depth restoration
  • matrix completion
  • view synthesis

ASJC Scopus subject areas

  • Media Technology
  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'Computational multi-view imaging with kinect'. Together they form a unique fingerprint.

  • Cite this