The elicitation of audiovisual steady-state responses: Multi-sensory signal congruity and phase effects

J. Jenkins, Ariane E. Rhone, William J. Idsardi, Jonathan Z. Simon, David Poeppel

Research output: Contribution to journalArticle

Abstract

Most ecologically natural sensory inputs are not limited to a single modality. While it is possible to use real ecological materials as experimental stimuli to investigate the neural basis of multi-sensory experience, parametric control of such tokens is limited. By using artificial bimodal stimuli composed of approximations to ecological signals, we aim to observe the interactions between putatively relevant stimulus attributes. Here we use MEG as an electrophysiological tool and employ as a measure the steady-state response (SSR), an experimental paradigm typically applied to unimodal signals. In this experiment we quantify the responses to a bimodal audio-visual signal with different degrees of temporal (phase) congruity, focusing on stimulus properties critical to audiovisual speech. An amplitude modulated auditory signal ('pseudo-speech') is paired with a radius-modulated ellipse ('pseudo-mouth'), with the envelope of low-frequency modulations occurring in phase or at offset phase values across modalities.We observe (i) that it is possible to elicit an SSR to bimodal signals; (ii) that bimodal signals exhibit greater response power than unimodal signals; and (iii) that the SSR power at specific harmonics and sensors differentially reflects the congruity between signal components. Importantly, we argue that effects found at the modulation frequency and second harmonic reflect differential aspects of neural coding of multisensory signals. The experimental paradigm facilitates a quantitative characterization of properties of multi-sensory speech and other bimodal computations.

Original languageEnglish (US)
Pages (from-to)134-148
Number of pages15
JournalBrain Topography
Volume24
Issue number2
DOIs
StatePublished - Jun 2011

Keywords

  • Audio-visual
  • Cross-modal
  • Magnetoencephalography
  • Multi-sensory
  • Speech

ASJC Scopus subject areas

  • Anatomy
  • Radiological and Ultrasound Technology
  • Radiology Nuclear Medicine and imaging
  • Neurology
  • Clinical Neurology

Fingerprint Dive into the research topics of 'The elicitation of audiovisual steady-state responses: Multi-sensory signal congruity and phase effects'. Together they form a unique fingerprint.

Cite this