Non-coplanar multiview sequence compression

This paper considers the problem of structure and motion estimation in multiview and stereoscopic tele-conferencing type sequences and its application for video sequence compression and for intermediate view generation. Generally, this type of sequence consists of one or several foreground objects (head and shoulder) and a more or less static background. By extracting the foreground objects and estimating the disparity field between a selected view and a reference view, we can compress the image-pair very efficiently. In the mean time, by using the estimated structure information, we can generate virtual viewpoints between views, which can be very helpful for tele-presence applications. We introduce an image alignment approach which can convert images captured from non-parallel camera geometry to coplanar like images, and this approach eases greatly the computational burden incurred by the non-parallel camera geometry where one must consider both the horizontal and vertical disparity.