Mouth shape detection and tracking using an active mesh

Yao Wang, Ru Shang Wang, Ouseb Lee, Tsuhan Chen, Homer H. Chen, Barry G. Haskell

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In this paper, we describe an approach to detecting and tracking certain feature points in the mouth region in a talking head sequence. These feature points are interconnected in a polygonal mesh so that the detection and tracking of these points is based on the information not only at these points but also in the surrounding elements. The detection of the nodes in an initial frame is accomplished by a feature detection algorithm. The tracking of these nodes in successive frames is obtained by deforming the mesh so that, when one mesh is warped to the other, the image patterns over corresponding elements in two meshes match with each other. This is accomplished by a modified Newton algorithm which iteratively minimized the error between the two images after mesh-based- warping. The numerical calculation involved in the optimization approach is simplified by using the concept of master elements and shape functions in the finite element method. This algorithm has been applied to a SIF resolution sequence, which contains fairly rapid mouth movement. Our simulation results show that this algorithm can locate and track the feature points in the mouth region quite accurately.

Original languageEnglish (US)
Title of host publicationProceedings of SPIE - The International Society for Optical Engineering
PublisherSociety of Photo-Optical Instrumentation Engineers
Pages1141-1152
Number of pages12
Edition2/-
ISBN (Print)0819418587
StatePublished - 1995
EventVisual Communications and Image Processing '95 - Taipei, Taiwan
Duration: May 24 1995May 26 1995

Publication series

NameProceedings of SPIE - The International Society for Optical Engineering
Number2/-
Volume2501
ISSN (Print)0277-786X

Other

OtherVisual Communications and Image Processing '95
CityTaipei, Taiwan
Period5/24/955/26/95

ASJC Scopus subject areas

  • Electronic, Optical and Magnetic Materials
  • Condensed Matter Physics
  • Computer Science Applications
  • Applied Mathematics
  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'Mouth shape detection and tracking using an active mesh'. Together they form a unique fingerprint.

  • Cite this

    Wang, Y., Wang, R. S., Lee, O., Chen, T., Chen, H. H., & Haskell, B. G. (1995). Mouth shape detection and tracking using an active mesh. In Proceedings of SPIE - The International Society for Optical Engineering (2/- ed., pp. 1141-1152). (Proceedings of SPIE - The International Society for Optical Engineering; Vol. 2501 , No. 2/-). Society of Photo-Optical Instrumentation Engineers.