Does audiovisual speech perception use information about facial configuration?

J. K. Hietanen*, P. Manninen, M. Sams, V. Surakka

*Corresponding author for this work

Research output: Contribution to journalArticleScientificpeer-review

15 Citations (Scopus)

Abstract

The McGurk illusion effectively demonstrates the audiovisual nature of speech perception. When an auditory syllable is dubbed onto an incongruous visual syllable the resulting percept is usually not either of the components, but their combination or fusion. The present experiment investigated the persistence of the McGurk effect when the facial configuration context of the audiovisual stimuli was manipulated. Two congruent and two incongruent audiovisual syllables were created from spoken /ipi/ and /iki/. These audiovisual tokens were uttered by seven facial configurations of five talkers. All facial configurations produced a clear McGurk effect (reported /iti/ for heard /ipi/ + seen /iki/), but the effect was significantly less when uttered by an asymmetrically scrambled face configuration. The results also showed significant differences in the persistence of the McGurk effect between different talkers. In sum, facial configural information can be used in the audiovisual integration of certain speech segments. This information is not necessary for the integration to occur, but the integration process can be disrupted by face stimuli violating the normal configural information.

Original languageEnglish
Pages (from-to)395-407
Number of pages13
JournalEuropean Journal of Cognitive Psychology
Volume13
Issue number3
DOIs
Publication statusPublished - 2001
MoE publication typeA1 Journal article-refereed

Fingerprint Dive into the research topics of 'Does audiovisual speech perception use information about facial configuration?'. Together they form a unique fingerprint.

  • Cite this