Linear integration of sensory evidence over space and time underlies face categorization

Gouki Okazawa, Long Sha, Roozbeh Kiani

Research output: Contribution to journalArticlepeer-review

Abstract

Visual object recognition relies on elaborate sensory processes that transform retinal inputs to object representations, but it also requires decision-making processes that read out object representations and function over prolonged time scales. The computational properties of these decision-making processes remain underexplored for object recognition. Here, we study these computations by developing a stochastic multifeature face categorization task. Using quantitative models and tight control of spatiotemporal visual information, we demonstrate that human subjects (five males, eight females) categorize faces through an integration process that first linearly adds the evidence conferred by task-relevant features over space to create aggregated momentary evidence and then linearly integrates it over time with minimum information loss. Discrimination of stimuli along different category boundaries (e.g., identity or expression of a face) is implemented by adjusting feature weights of spatial integration. This linear but flexible integration process over space and time bridges past studies on simple perceptual decisions to complex object recognition behavior. SIGNIFICANCE STATEMENT Although simple perceptual decision-making such as discrimination of random dot motion has been successfully explained as accumulation of sensory evidence, we lack rigorous experimental paradigms to study the mechanisms underlying complex perceptual decision-making such as discrimination of naturalistic faces. We develop a stochastic multifeature face categorization task as a systematic approach to quantify the properties and potential limitations of the decision-making processes during object recognition. We show that human face categorization could be modeled as a linear integration of sensory evidence over space and time. Our framework to study object recognition as a spatiotemporal integration process is broadly applicable to other object categories and bridges past studies of object recognition and perceptual decision-making.

Original languageEnglish (US)
Pages (from-to)7876-7893
Number of pages18
JournalJournal of Neuroscience
Volume41
Issue number37
DOIs
StatePublished - Sep 15 2021

Keywords

  • Bounded accumulation of evidence
  • Face recognition
  • Feature combination
  • Flexible decision making
  • Linear spatiotemporal integration
  • Psychophysics
  • Reverse correlation
  • Humans
  • Male
  • Neuropsychological Tests
  • Eye Movements/physiology
  • Reaction Time/physiology
  • Decision Making/physiology
  • Female
  • Pattern Recognition, Visual/physiology
  • Photic Stimulation
  • Facial Recognition/physiology

ASJC Scopus subject areas

  • Neuroscience(all)

Fingerprint

Dive into the research topics of 'Linear integration of sensory evidence over space and time underlies face categorization'. Together they form a unique fingerprint.

Cite this