Diffusion operators for multimodal data analysis

Tal Shnitzer, Roy R. Lederman, Gi Ren Liu, Ronen Talmon, Hau Tieng Wu

Research output: Chapter in Book/Report/Conference proceedingChapter


In this chapter, we present a Manifold Learning viewpoint on the analysis of data arising from multiple modalities. We assume that the high-dimensional multimodal data lie on underlying low-dimensional manifolds and devise a new data-driven representation that accommodates this inherent structure. Based on diffusion geometry, we present three composite operators, facilitating different aspects of fusion of information from different modalities in different settings. These operators are shown to recover the common structures and the differences between modalities in terms of their intrinsic geometry and allow for the construction of data-driven representations which capture these characteristics. The properties of these operators are demonstrated in four applications: recovery of the common variable in two camera views, shape analysis, foetal heart rate identification and sleep dynamics assessment.

Original languageEnglish (US)
Title of host publicationProcessing, Analyzing and Learning of Images, Shapes, and Forms
Subtitle of host publicationPart 2
EditorsRon Kimmel, Xue-Cheng Tai
PublisherElsevier B.V.
Number of pages39
ISBN (Print)9780444641403
StatePublished - 2019

Publication series

NameHandbook of Numerical Analysis
ISSN (Print)1570-8659


  • 57M50
  • 57R40
  • 62-07
  • 62M15
  • Alternating Diffusion
  • Common variable
  • Diffusion Maps
  • Manifold Learning
  • Multimodal data
  • Sensor fusion
  • Shape differences

ASJC Scopus subject areas

  • Numerical Analysis
  • Modeling and Simulation
  • Computational Mathematics
  • Applied Mathematics


Dive into the research topics of 'Diffusion operators for multimodal data analysis'. Together they form a unique fingerprint.

Cite this