Film Mood: Assessment, Analysis, and Computational Modeling

Film is an affective art form. Mainstream films in particular are carefully designed to provide viewers with affective experiences, and while the exact nature of this experience differs from film to film, the general strategy is notably consistent: films tell engaging stories populated by empathetic characters and presented cinematically in a way that grabs the viewers’ attention, holds them in suspense, and elicits various emotions. Together, these narrative and aesthetic means lend each film scene a distinct mood, which describes the affective expression of the scene. By conveying a film’s intent to evoke a certain affective experience, film mood can provide insight into the ways in which films affect viewers. It can also be used to describe and classify films based on their affective properties. As such, the concept of film mood may prove helpful not only from the perspective of film studies, but computer science as well, in which recent efforts have sought to develop methods to estimate the affective content of films based on features detected computationally from the film material. This dissertation studies film mood from the dual perspectives of cognitive film studies and computer science; that is, both perceptually and computationally. It has three main objectives: to determine which factors should be considered when assessing film mood, to examine some of the perceptual properties of film mood, and to investigate the extent to which ratings of film mood can be modeled with computational features. To this end, two user studies were conducted, and the data collected in the studies was used for both a perceptual analysis of film mood and for computational modeling of mood ratings. The modeling involved both commonly used low-level computational features as well as state-of-the-art high-level features. The results showed that film mood was an intuitive concept for viewers to assess and robust against individual differences between assessors. They also showed that the perceived intersity of film scenes tended to follow a rise–plateau–fall structure, that extremely negative and positive moods were most common in scenes where music had the most prominent role on the soundtrack, and that the stylistic attributes of a scene were most strongly related to how energetic the scene was perceived to be. Lastly, the results showed that the low- and high-level computational features complemented one another, providing unique contributions to mood modeling. Together, the results indicate that the concept of film mood is a useful one in terms of both film studies and computer science. By illuminating some of the ways in which film mood is related to both perceptual attributes and computational features, the dissertation contributes to recent efforts in these two fields to increase understanding about the affective experience of film and to improve the characterization of film contents by incorporating affective properties into the description.