Affect in Multimedia: Benchmarking Violent Scenes Detection

Mihai Gabriel Constantin, Liviu Daniel Stefan, Bogdan Ionescu, Claire Helene Demarty, Mats Sjoberg, Markus Schedl, Guillaume Gravier

Research output: Contribution to journalArticleScientificpeer-review

1 Citation (Scopus)

Abstract

In this paper, we report on the creation of a publicly available, common evaluation framework, for Violent Scenes Detection (VSD) in Hollywood and YouTube videos. We propose a robust data set, the VSD96, with more than 96 hours of video of various genres, annotations at different levels of detail (e.g., shot-level, segment-level), annotations of mid-level concepts (e.g., blood, fire), various pre-computed multi-modal descriptors, and over 230 system output results as baselines. This is the most comprehensive dataset available to this date tailored to the VSD task, and was extensively validated during the MediaEval benchmarking campaigns. Furthermore, we provide an in-depth analysis of the crucial components of VSD algorithms, by reviewing the capabilities and the evolution of existing systems (e.g., overall trends and outliers, influence of the employed features and fusion techniques). Finally, we discuss the possibility of going beyond state-of-the-art performance via an ad-hoc late fusion approach. Experimentation is carried out on the VSD96 data. The increasing number of publications using the VSD96 data underline the importance of the topic. The presented and published resources are a practitioner's guide and also a strong baseline to overcome, which will help researchers for the coming years in analyzing aspects of audio-visual affect and violence detection in movies and videos.

Original languageEnglish
JournalIEEE Transactions on Affective Computing
DOIs
Publication statusE-pub ahead of print - 1 Jan 2020
MoE publication typeA1 Journal article-refereed

Keywords

  • Benchmark testing
  • benchmarking
  • literature review
  • Machine learning
  • Market research
  • Motion pictures
  • multi-modal content description
  • Task analysis
  • Videos
  • violent scenes detection
  • VSD96 data set
  • YouTube

Fingerprint Dive into the research topics of 'Affect in Multimedia: Benchmarking Violent Scenes Detection'. Together they form a unique fingerprint.

Cite this