Discriminating languages by speech-reading

Salvador Soto-Faraco, Jordi Navarra, Whitnev M. Weikum, Athena Vouloumanos, Núria Sebastián-Gallés, Janet F. Werker

Research output: Contribution to journalArticle

Abstract

The goal of this study was to explore the ability to discriminate languages using the visual correlates of speech (i.e., speech-reading). Participants were presented with silent video clips of an actor pronouncing two sentences (in Catalan and/or Spanish) and were asked to judge whether the sentences were in the same language or in different languages. Our results established that Spanish-Catalan bilingual speakers could discriminate running speech from their two languages on the basis of visual cues alone (Experiment 1). However, we found that this ability was critically restricted by linguistic experience, since Italian and English speakers who were unfamiliar with the test languages could not successfully discriminate the stimuli (Experiment 2). A test of Spanish monolingual speakers revealed that knowledge of only one of the two test languages was sufficient to achieve the discrimination, although at a lower level of accuracy than that seen in bilingual speakers (Experiment 3). Finally, we evaluated the ability to identify the language by speech-reading particularly distinctive words (Experiment 4). The results obtained are in accord with recent proposals arguing that the visual speech signal is rich in informational content, above and beyond what traditional accounts based solely on visemic confusion matrices would predict.

Original languageEnglish (US)
Pages (from-to)218-231
Number of pages14
JournalPerception and Psychophysics
Volume69
Issue number2
DOIs
StatePublished - Feb 2007

ASJC Scopus subject areas

  • Experimental and Cognitive Psychology
  • Sensory Systems
  • Psychology(all)

Fingerprint Dive into the research topics of 'Discriminating languages by speech-reading'. Together they form a unique fingerprint.

Cite this