When to Laugh and How Hard? A Multimodal Approach to Detecting Humor and Its Intensity

Khalid Alnajjar; Mika Hämäläinen; Jörg Tiedemann; Jorma Laaksonen; Mikko Kurimo

When to Laugh and How Hard? A Multimodal Approach to Detecting Humor and Its Intensity

Khalid Alnajjar^*, Mika Hämäläinen, Jörg Tiedemann, Jorma Laaksonen, Mikko Kurimo

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Scientific › peer-review

9 Downloads (Pure)

Abstract

Prerecorded laughter accompanying dialog in comedy TV shows encourages the audience to laugh by clearly marking humorous moments in the show. We present an approach for automatically detecting humor in the Friends TV show using multimodal data. Our model is capable of recognizing whether an utterance is humorous or not and assess the intensity of it. We use the prerecorded laughter in the show as annotation as it marks humor and the length of the audience’s laughter tells us how funny a given joke is. We evaluate the model on episodes the model has not been exposed to during the training phase. Our results show that the model is capable of correctly detecting whether an utterance is humorous 78% of the time and how long the audience’s laughter reaction should last with a mean absolute error of 600 milliseconds.

Original language	English
Title of host publication	Proceedings of the 29th International Conference on Computational Linguistics
Publisher	International Committee on Computational Linguistics
Pages	6875-6886
Number of pages	12
Publication status	Published - Oct 2022
MoE publication type	A4 Article in a conference publication
Event	International Conference on Computational Linguistics - Gyeongju, Korea, Republic of Duration: 12 Oct 2022 → 17 Oct 2022 https://coling2022.org/

Publication series

Name	Proceedings of the International Conference on Computational Linguistics
Publisher	International Committee on Computational Linguistics
ISSN (Electronic)	2951-2093

Conference

Conference	International Conference on Computational Linguistics
Country/Territory	Korea, Republic of
City	Gyeongju
Period	12/10/2022 → 17/10/2022
Internet address	https://coling2022.org/

Access to Document

2022.coling-1.598Final published version, 7.2 MB

OpenUrl availability

Full text

USSEE: Understanding speech and scene with ears and eyes (USSEE)
Laaksonen, J., Pehlivan Tort, S., Wang, T., Guo, Z., Tiwari, H. & Arora, P.
01/01/2022 → 31/12/2024
Project: Academy of Finland: Other research funding

Science-IT
Mikko Hakala (Manager)
School of Science
Facility/equipment: Facility

Cite this

Alnajjar, K., Hämäläinen, M., Tiedemann, J., Laaksonen, J., & Kurimo, M. (2022). When to Laugh and How Hard? A Multimodal Approach to Detecting Humor and Its Intensity. In Proceedings of the 29th International Conference on Computational Linguistics (pp. 6875-6886). (Proceedings of the International Conference on Computational Linguistics). International Committee on Computational Linguistics. https://aclanthology.org/2022.coling-1.598

Alnajjar, Khalid ; Hämäläinen, Mika ; Tiedemann, Jörg et al. / When to Laugh and How Hard? A Multimodal Approach to Detecting Humor and Its Intensity. Proceedings of the 29th International Conference on Computational Linguistics. International Committee on Computational Linguistics, 2022. pp. 6875-6886 (Proceedings of the International Conference on Computational Linguistics).

@inproceedings{54cf34207eb142708061a2b6a4a22def,

title = "When to Laugh and How Hard? A Multimodal Approach to Detecting Humor and Its Intensity",

abstract = "Prerecorded laughter accompanying dialog in comedy TV shows encourages the audience to laugh by clearly marking humorous moments in the show. We present an approach for automatically detecting humor in the Friends TV show using multimodal data. Our model is capable of recognizing whether an utterance is humorous or not and assess the intensity of it. We use the prerecorded laughter in the show as annotation as it marks humor and the length of the audience{\textquoteright}s laughter tells us how funny a given joke is. We evaluate the model on episodes the model has not been exposed to during the training phase. Our results show that the model is capable of correctly detecting whether an utterance is humorous 78% of the time and how long the audience{\textquoteright}s laughter reaction should last with a mean absolute error of 600 milliseconds.",

author = "Khalid Alnajjar and Mika H{\"a}m{\"a}l{\"a}inen and J{\"o}rg Tiedemann and Jorma Laaksonen and Mikko Kurimo",

year = "2022",

month = oct,

language = "English",

series = "Proceedings of the International Conference on Computational Linguistics",

publisher = "International Committee on Computational Linguistics",

pages = "6875--6886",

booktitle = "Proceedings of the 29th International Conference on Computational Linguistics",

address = "Czech Republic",

note = "International Conference on Computational Linguistics ; Conference date: 12-10-2022 Through 17-10-2022",

url = "https://coling2022.org/",

}

Alnajjar, K, Hämäläinen, M, Tiedemann, J, Laaksonen, J & Kurimo, M 2022, When to Laugh and How Hard? A Multimodal Approach to Detecting Humor and Its Intensity. in Proceedings of the 29th International Conference on Computational Linguistics. Proceedings of the International Conference on Computational Linguistics, International Committee on Computational Linguistics, pp. 6875-6886, International Conference on Computational Linguistics, Gyeongju, Korea, Republic of, 12/10/2022. <https://aclanthology.org/2022.coling-1.598>

When to Laugh and How Hard? A Multimodal Approach to Detecting Humor and Its Intensity. / Alnajjar, Khalid; Hämäläinen, Mika; Tiedemann, Jörg et al.
Proceedings of the 29th International Conference on Computational Linguistics. International Committee on Computational Linguistics, 2022. p. 6875-6886 (Proceedings of the International Conference on Computational Linguistics).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Scientific › peer-review

TY - GEN

T1 - When to Laugh and How Hard? A Multimodal Approach to Detecting Humor and Its Intensity

AU - Alnajjar, Khalid

AU - Hämäläinen, Mika

AU - Tiedemann, Jörg

AU - Laaksonen, Jorma

AU - Kurimo, Mikko

PY - 2022/10

Y1 - 2022/10

N2 - Prerecorded laughter accompanying dialog in comedy TV shows encourages the audience to laugh by clearly marking humorous moments in the show. We present an approach for automatically detecting humor in the Friends TV show using multimodal data. Our model is capable of recognizing whether an utterance is humorous or not and assess the intensity of it. We use the prerecorded laughter in the show as annotation as it marks humor and the length of the audience’s laughter tells us how funny a given joke is. We evaluate the model on episodes the model has not been exposed to during the training phase. Our results show that the model is capable of correctly detecting whether an utterance is humorous 78% of the time and how long the audience’s laughter reaction should last with a mean absolute error of 600 milliseconds.

AB - Prerecorded laughter accompanying dialog in comedy TV shows encourages the audience to laugh by clearly marking humorous moments in the show. We present an approach for automatically detecting humor in the Friends TV show using multimodal data. Our model is capable of recognizing whether an utterance is humorous or not and assess the intensity of it. We use the prerecorded laughter in the show as annotation as it marks humor and the length of the audience’s laughter tells us how funny a given joke is. We evaluate the model on episodes the model has not been exposed to during the training phase. Our results show that the model is capable of correctly detecting whether an utterance is humorous 78% of the time and how long the audience’s laughter reaction should last with a mean absolute error of 600 milliseconds.

M3 - Conference contribution

T3 - Proceedings of the International Conference on Computational Linguistics

SP - 6875

EP - 6886

BT - Proceedings of the 29th International Conference on Computational Linguistics

PB - International Committee on Computational Linguistics

T2 - International Conference on Computational Linguistics

Y2 - 12 October 2022 through 17 October 2022

ER -

Alnajjar K, Hämäläinen M, Tiedemann J, Laaksonen J , Kurimo M. When to Laugh and How Hard? A Multimodal Approach to Detecting Humor and Its Intensity. In Proceedings of the 29th International Conference on Computational Linguistics. International Committee on Computational Linguistics. 2022. p. 6875-6886. (Proceedings of the International Conference on Computational Linguistics).

When to Laugh and How Hard? A Multimodal Approach to Detecting Humor and Its Intensity

Abstract

Publication series

Conference

Access to Document

OpenUrl availability

Projects

USSEE: Understanding speech and scene with ears and eyes (USSEE)

Equipment

Science-IT

Cite this