Computational and perceptual determinants of film mood in different types of scenes

Jussi Tarvainen, Jorma Laaksonen, Tapio Takala

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingsScientificpeer-review

1 Citation (Scopus)


Films seek to elicit emotions in viewers by infusing the story they tell with an affective character or tone - in a word, a mood. In content-based multimedia analysis, considerable effort has been made to develop methods to estimate film affect computationally. However, results have been hampered by a tendency to classify film scenes either by genre or not at all, while other potentially helpful classification methods have been neglected. In this study, we investigated the quantitative determinants of film mood across different types of scenes. We first collected style and mood ratings for 50 film scenes, which we classified by their location, time of day, and their use of dialogue and music. We then investigated whether the viewers rated the mood (in terms of hedonic tone, energetic arousal, and tense arousal) of various scene types differently, and how well perceptual stylistic attributes as well as low- and high-level computational features correlated with the mood ratings. We found that the mood ratings and their quantitative determinants differed across the scene types. We also found that the energetic arousal ratings were associated with the stylistic attributes and their corresponding low-level features, while hedonic tone and tense arousal were associated with high-level features related to the emotional expression in faces, dialogue, and music. The study contributes to ongoing efforts to estimate film affect computationally in showing that results can be improved by utilizing both low- and high-level features and by considering different scene types separately.
Original languageEnglish
Title of host publicationProceedings - 2017 IEEE International Symposium on Multimedia, ISM 2017
ISBN (Print)9781538629376, 9781538629369
Publication statusPublished - Dec 2017
MoE publication typeA4 Conference publication
EventIEEE International Symposium on Multimedia - Taichung, Taiwan, Republic of China
Duration: 11 Dec 201713 Jan 2018
Conference number: 19


ConferenceIEEE International Symposium on Multimedia
Abbreviated titleISM
Country/TerritoryTaiwan, Republic of China


  • film
  • affect
  • mood
  • style
  • content-based analysis


Dive into the research topics of 'Computational and perceptual determinants of film mood in different types of scenes'. Together they form a unique fingerprint.

Cite this