The effect of linguistic and visual salience in visual world studies

Federica Cavicchio, David Melcher, Massimo Poesio

Research output: Contribution to journalArticle

Abstract

Research using the visual world paradigm has demonstrated that visual input has a rapid effect on language interpretation tasks such as reference resolution and, conversely, that linguistic material-including verbs, prepositions and adjectives-can influence fixations to potential referents. More recent research has started to explore how this effect of linguistic input on fixations is mediated by properties of the visual stimulus, in particular by visual salience. In the present study we further explored the role of salience in the visual world paradigm manipulating language-driven salience and visual salience. Specifically, we tested how linguistic salience (i.e., the greater accessibility of linguistically introduced entities) and visual salience (bottom-up attention grabbing visual aspects) interact. We recorded participants' eye-movements during a MapTask, asking them to look from landmark to landmark displayed upon a map while hearing direction-giving instructions. The landmarks were of comparable size and color, except in the Visual Salience condition, in which one landmark had been made more visually salient. In the Linguistic Salience conditions, the instructions included references to an object not on the map. Response times and fixations were recorded. Visual Salience influenced the time course of fixations at both the beginning and the end of the trial but did not show a significant effect on response times. Linguistic Salience reduced response times and increased fixations to landmarks when they were associated to a Linguistic Salient entity not present itself on the map. When the target landmark was both visually and linguistically salient, it was fixated longer, but fixations were quicker when the target item was linguistically salient only. Our results suggest that the two types of salience work in parallel and that linguistic salience affects fixations even when the entity is not visually present.

Original languageEnglish (US)
Article numberArticle 176
JournalFrontiers in Psychology
Volume5
Issue numberMAR
DOIs
StatePublished - 2014

Keywords

  • Centering theory
  • Linguistic salience
  • Saliency map
  • Visual salience
  • Visual world paradigm

ASJC Scopus subject areas

  • Psychology(all)

Fingerprint Dive into the research topics of 'The effect of linguistic and visual salience in visual world studies'. Together they form a unique fingerprint.

  • Cite this