Multiview video sequence analysis, compression, and virtual viewpoint synthesis

Ru Shang Wang, Yao Wang

Research output: Contribution to journalArticlepeer-review

Abstract

This paper considers the problem of structure and motion estimation in multiview teleconferencing-type sequences and its application for video-sequence compression and intermediate-view generation. First, we introduce a new approach for structure estimation from a stereo pair acquired by two parallel cameras. It is based on a 2-D mesh representation of both views of the imaged scene and a parameterization of the structure information by the disparity between corresponding nodes in the image pair. Next, we describe a novel image alignment approach which can convert images captured using nonparallel cameras to coplanar-like images. This approach greatly eases the computational burden incurred by the nonparallel camera geometry, where one must consider both horizontal and vertical disparities. Finally, we present a coder for multiview sequences, which exploits the proposed alignment and structure estimation algorithm. By extracting the foreground objects and estimating the disparity field between a selected view and a reference view, the coder can compress the image pair very efficiently. In the meantime, by using the coded structure information, the decoder can generate virtual viewpoints between decoded views, which can be very helpful for telepresence applications.

Original languageEnglish (US)
Pages (from-to)397-410
Number of pages14
JournalIEEE Transactions on Circuits and Systems for Video Technology
Volume10
Issue number3
DOIs
StatePublished - 2000

ASJC Scopus subject areas

  • Media Technology
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'Multiview video sequence analysis, compression, and virtual viewpoint synthesis'. Together they form a unique fingerprint.

Cite this