Modulation of Brain Activity by Selective Attention to Audiovisual Dialogues

Alina Leminen*, Maxime Verwoert, Mona Moisala, Viljami Salmela, Patrik Wikman, Kimmo Alho

*Corresponding author for this work

Research output: Contribution to journalArticle

1 Citation (Scopus)
18 Downloads (Pure)

Abstract

In real-life noisy situations, we can selectively attend to conversations in the presence of irrelevant voices, but neurocognitive mechanisms in such natural listening situations remain largely unexplored. Previous research has shown distributed activity in the mid superior temporal gyrus (STG) and sulcus (STS) while listening to speech and human voices, in the posterior STS and fusiform gyrus when combining auditory, visual and linguistic information, as well as in left-hemisphere temporal and frontal cortical areas during comprehension. In the present functional magnetic resonance imaging (fMRI) study, we investigated how selective attention modulates neural responses to naturalistic audiovisual dialogues. Our healthy adult participants (N = 15) selectively attended to video-taped dialogues between a man and woman in the presence of irrelevant continuous speech in the background. We modulated the auditory quality of dialogues with noise vocoding and their visual quality by masking speech-related facial movements. Both increased auditory quality and increased visual quality were associated with bilateral activity enhancements in the STG/STS. In addition, decreased audiovisual stimulus quality elicited enhanced fronto-parietal activity, presumably reflecting increased attentional demands. Finally, attention to the dialogues, in relation to a control task where a fixation cross was attended and the dialogue ignored, yielded enhanced activity in the left planum polare, angular gyrus, the right temporal pole, as well as in the orbitofrontal/ventromedial prefrontal cortex and posterior cingulate gyrus. Our findings suggest that naturalistic conversations effectively engage participants and reveal brain networks related to social perception in addition to speech and semantic processing networks.

Original languageEnglish
Article number436
Pages (from-to)1-16
Number of pages16
JournalFrontiers in Neuroscience
Volume14
DOIs
Publication statusPublished - 12 May 2020
MoE publication typeA1 Journal article-refereed

Keywords

  • audiovisual integration
  • fMRI
  • noise vocoding
  • selective attention
  • social perception
  • speech
  • visual speech

Fingerprint Dive into the research topics of 'Modulation of Brain Activity by Selective Attention to Audiovisual Dialogues'. Together they form a unique fingerprint.

Cite this