Effects of spectro-temporal asynchrony in auditory and auditory-visual speech processing

Throughout his career, Ira Hirsh studied and published articles and books pertaining to many aspects of the auditory system. These included sound conduction in the ear, cochlear mechanics, masking, auditory localization, psychoacoustic behavior in animals, speech perception, medical and audiological applications, coupling between psychophysics and physiology, and ecological acoustics. However, it is Hirsh's work on auditory timing of simple and complex rhythmic patterns, the backbone of speech and music, that are at the heart of his more recent work. In this article, we report on several aspects of temporal processing of speech signals, both within and across sensory systems. Data are presented on perceived simultaneity and intelligibility of auditory and auditory-visual speech stimuli in which stimulus components are presented either synchronously or asynchronously. Differences in the symmetry and shape of temporal windows derived from these datasets are highlighted. Results show two distinct ranges for temporal integration for speech processing; one relatively short window, ∼ 40 milliseconds, and the other much longer, around 250 milliseconds. In the case of auditory-visual speech processing, the temporal window is highly asymmetric, strongly favoring conditions where the visual stimulus precedes the acoustic stimulus.