The semiotics of the message and the messenger: How nonverbal communication affects fairness perception

Michiel Spapé*, Ville Harjunen, Imtiaj Ahmed, Giulio Jacucci, Niklas Ravaja

*Corresponding author for this work

Research output: Contribution to journalArticleScientificpeer-review

2 Citations (Scopus)
115 Downloads (Pure)


Nonverbal communication determines much of how we perceive explicit, verbal messages. Facial expressions and social touch, for example, influence affinity and conformity. To understand the interaction between nonverbal and verbal information, we studied how the psychophysiological time-course of semiotics—the decoding of the meaning of a message—is altered by interpersonal touch and facial expressions. A virtual-reality-based economic decision-making game, ultimatum, was used to investigate how participants perceived, and responded to, financial offers of variable levels of fairness. In line with previous studies, unfair offers evoked medial frontal negativity (MFN) within the N2 time window, which has been interpreted as reflecting an emotional reaction to violated social norms. Contrary to this emotional interpretation of the MFN, however, nonverbal signals did not modulate the MFN component, only affecting fairness perception during the P3 component. This suggests that the nonverbal context affects the late, but not the early, stage of fairness perception. We discuss the implications of the semiotics of the message and the messenger as a process by which parallel information sources of “who says what” are integrated in reverse order: of the message, then the messenger.

Original languageEnglish
Pages (from-to)1259-1272
JournalCognitive, Affective and Behavioral Neuroscience
Issue number5
Early online date9 Jul 2019
Publication statusPublished - Oct 2019
MoE publication typeA1 Journal article-refereed


  • Economic decision-making
  • EEG
  • Emotional expressions
  • ERP
  • MFN
  • Neurosemiotics
  • Nonverbal communication
  • Touch

Fingerprint Dive into the research topics of 'The semiotics of the message and the messenger: How nonverbal communication affects fairness perception'. Together they form a unique fingerprint.

Cite this