Computational and perceptual determinants of film mood in different types of scenes

Films seek to elicit emotions in viewers by infusing the story they tell with an affective character or tone - in a word, a mood. In content-based multimedia analysis, considerable effort has been made to develop methods to estimate film affect computationally. However, results have been hampered by a tendency to classify film scenes either by genre or not at all, while other potentially helpful classification methods have been neglected. In this study, we investigated the quantitative determinants of film mood across different types of scenes. We first collected style and mood ratings for 50 film scenes, which we classified by their location, time of day, and their use of dialogue and music. We then investigated whether the viewers rated the mood (in terms of hedonic tone, energetic arousal, and tense arousal) of various scene types differently, and how well perceptual stylistic attributes as well as low- and high-level computational features correlated with the mood ratings. We found that the mood ratings and their quantitative determinants differed across the scene types. We also found that the energetic arousal ratings were associated with the stylistic attributes and their corresponding low-level features, while hedonic tone and tense arousal were associated with high-level features related to the emotional expression in faces, dialogue, and music. The study contributes to ongoing efforts to estimate film affect computationally in showing that results can be improved by utilizing both low- and high-level features and by considering different scene types separately.