Reconstructing Speech Stimuli From Human Auditory Cortex Activity Using a WaveNet Approach

Ran Wang, Yao Wang, Adeen Flinker

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The superior temporal gyrus (STG) region of cortex critically contributes to speech recognition. In this work, we show that a proposed WaveNet, with limited available data, is able to reconstruct speech stimuli from STG intracranial recordings. We further investigate the impulse response of the fitted model for each recording electrode and observe phoneme level temporospectral tuning properties for the recorded area of cortex. This discovery is consistent with previous studies implicating the posterior STG (pSTG) in a phonetic representation of speech and provides detailed acoustic features that certain electrode sites possibly extract during speech recognition.

Original languageEnglish (US)
Title of host publication2018 IEEE Signal Processing in Medicine and Biology Symposium, SPMB 2018 - Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781538659168
DOIs
StatePublished - Jan 16 2019
Event2018 IEEE Signal Processing in Medicine and Biology Symposium, SPMB 2018 - Philadelphia, United States
Duration: Dec 1 2018 → …

Publication series

Name2018 IEEE Signal Processing in Medicine and Biology Symposium, SPMB 2018 - Proceedings

Conference

Conference2018 IEEE Signal Processing in Medicine and Biology Symposium, SPMB 2018
CountryUnited States
CityPhiladelphia
Period12/1/18 → …

ASJC Scopus subject areas

  • Signal Processing
  • Health Informatics

Fingerprint Dive into the research topics of 'Reconstructing Speech Stimuli From Human Auditory Cortex Activity Using a WaveNet Approach'. Together they form a unique fingerprint.

  • Cite this

    Wang, R., Wang, Y., & Flinker, A. (2019). Reconstructing Speech Stimuli From Human Auditory Cortex Activity Using a WaveNet Approach. In 2018 IEEE Signal Processing in Medicine and Biology Symposium, SPMB 2018 - Proceedings [8615605] (2018 IEEE Signal Processing in Medicine and Biology Symposium, SPMB 2018 - Proceedings). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/SPMB.2018.8615605