Using crowdsourced listeners’ ratings to measure speech changes in hypokinetic dysarthria: A proof-of-concept study

Christopher Nightingale, Michelle Swartz, Lorraine Olson Ramig, Tara McAllister

Research output: Contribution to journalArticlepeer-review


Purpose: Interventions for speech disorders aim to produce changes that are not only acoustically measurable or perceptible to trained professionals but are also apparent to naive listeners. Due to challenges associated with obtaining ratings from suitably large listener samples, however, few studies currently evaluate speech interventions by this criterion. Online crowdsourcing technologies could enhance the measurement of intervention effects by making it easier to obtain real-world listeners’ ratings. Method: Stimuli, drawn from a published study by Sapir et al. (“Effects of intensive voice treatment (Lee Silverman Voice Treatment [LSVT]) on vowel articulation in dysarthric individuals with idiopathic Parkinson disease: Acoustic and perceptual findings” in Journal of Speech, Language, and Hearing Research, 50(4), 2007), were words produced by individuals who received intensive treatment (LSVT LOUD) for hypokinetic dysarthria secondary to Parkinson’s disease. Thirty-six online naive listeners heard randomly ordered pairs of words elicited pre-and posttreatment and reported which they perceived as “more clearly articulated.” Results: Mixed-effects logistic regression indicated that words elicited posttreatment were significantly more likely to be rated “more clear.” Across individuals, acoustically measured magnitude of change was significantly correlated with pre–post difference in listener ratings. Conclusions: These results partly replicate the findings of Sapir et al. (2007) and demonstrate that their acoustically measured changes are detectable by everyday listeners. This supports the viability of using crowdsourcing to obtain more functionally relevant measures of change in clinical speech samples.

Original languageEnglish (US)
Pages (from-to)873-882
Number of pages10
JournalAmerican journal of speech-language pathology
Issue number2
StatePublished - May 2020


  • Crowdsourcing
  • Dysarthria/diagnosis
  • Humans
  • Speech
  • Speech Acoustics
  • Speech Intelligibility
  • Speech Perception
  • Speech Production Measurement
  • Speech Therapy

ASJC Scopus subject areas

  • Developmental and Educational Psychology
  • Speech and Hearing
  • Otorhinolaryngology
  • Linguistics and Language


Dive into the research topics of 'Using crowdsourced listeners’ ratings to measure speech changes in hypokinetic dysarthria: A proof-of-concept study'. Together they form a unique fingerprint.

Cite this