Combining Textual and Visual Modeling for Predicting Media Memorability

Alison Reboud; Ismail Harrando; Jorma Laaksonen; Danny Francis; Raphaël Troncy; Hector Laria Mantecon

Combining Textual and Visual Modeling for Predicting Media Memorability

Alison Reboud, Ismail Harrando, Jorma Laaksonen, Danny Francis, Raphaël Troncy, Hector Laria Mantecon

EURECOM

Research output: Chapter in Book/Report/Conference proceeding › Chapter › Scientific

2 Citations (Scopus)

114 Downloads (Pure)

Abstract

This paper describes a multimodal approach proposed by the MeMAD team for the MediaEval 2019 “Predicting Media memorability” task. Our best approach is a weighted average method combining predictions made separately from visual and textual representations of videos. In particular, we augmented the provided textual descriptions with automatically generated deep captions. For long term
memorability, we obtained better scores using the short term predictions rather than the long term ones. Our best model achieves Spearman scores of 0.522 and 0.277 respectively for the short and long term predictions tasks.

Original language	English
Title of host publication	Working Notes Proceedings of the MediaEval 2019 Workshop, Sophia Antipolis, France, 27-30 October 2019
Publisher	CEUR
Publication status	Published - 27 Oct 2019
MoE publication type	B2 Book section
Event	Multimedia Benchmark Workshop - Sophia Antipolis, France Duration: 27 Oct 2019 → 30 Oct 2019

Publication series

Name	CEUR Workshop Proceedings
Publisher	CEUR
Volume	2670
ISSN (Electronic)	1613-0073

Workshop

Workshop	Multimedia Benchmark Workshop
Abbreviated title	MediaEval
Country/Territory	France
City	Sophia Antipolis
Period	27/10/2019 → 30/10/2019

Access to Document

Reboud_Combining.MediaEval_19_paper_26Final published version, 689 KBLicence: CC BY

http://ceur-ws.org/Vol-2670/MediaEval_19_paper_26.pdf

Cite this

Reboud, A., Harrando, I., Laaksonen, J., Francis, D., Troncy, R., & Laria Mantecon, H. (2019). Combining Textual and Visual Modeling for Predicting Media Memorability. In Working Notes Proceedings of the MediaEval 2019 Workshop, Sophia Antipolis, France, 27-30 October 2019 (CEUR Workshop Proceedings; Vol. 2670). CEUR. http://ceur-ws.org/Vol-2670/MediaEval_19_paper_26.pdf

@inbook{0a76dbe617d84b8da8c34f3f60d92c89,

title = "Combining Textual and Visual Modeling for Predicting Media Memorability",

abstract = "This paper describes a multimodal approach proposed by the MeMAD team for the MediaEval 2019 “Predicting Media memorability” task. Our best approach is a weighted average method combining predictions made separately from visual and textual representations of videos. In particular, we augmented the provided textual descriptions with automatically generated deep captions. For long termmemorability, we obtained better scores using the short term predictions rather than the long term ones. Our best model achieves Spearman scores of 0.522 and 0.277 respectively for the short and long term predictions tasks.",

author = "Alison Reboud and Ismail Harrando and Jorma Laaksonen and Danny Francis and Rapha{\"e}l Troncy and {Laria Mantecon}, Hector",

note = "| openaire: EC/H2020/780069/EU//MeMAD ; Multimedia Benchmark Workshop, MediaEval ; Conference date: 27-10-2019 Through 30-10-2019",

year = "2019",

month = oct,

day = "27",

language = "English",

series = "CEUR Workshop Proceedings",

publisher = "CEUR",

booktitle = "Working Notes Proceedings of the MediaEval 2019 Workshop, Sophia Antipolis, France, 27-30 October 2019",

address = "Germany",

}

Reboud, A, Harrando, I, Laaksonen, J, Francis, D, Troncy, R & Laria Mantecon, H 2019, Combining Textual and Visual Modeling for Predicting Media Memorability. in Working Notes Proceedings of the MediaEval 2019 Workshop, Sophia Antipolis, France, 27-30 October 2019. CEUR Workshop Proceedings, vol. 2670, CEUR, Multimedia Benchmark Workshop, Sophia Antipolis, France, 27/10/2019. <http://ceur-ws.org/Vol-2670/MediaEval_19_paper_26.pdf>

Combining Textual and Visual Modeling for Predicting Media Memorability. / Reboud, Alison; Harrando, Ismail; Laaksonen, Jorma et al.
Working Notes Proceedings of the MediaEval 2019 Workshop, Sophia Antipolis, France, 27-30 October 2019. CEUR, 2019. (CEUR Workshop Proceedings; Vol. 2670).

Research output: Chapter in Book/Report/Conference proceeding › Chapter › Scientific

TY - CHAP

T1 - Combining Textual and Visual Modeling for Predicting Media Memorability

AU - Reboud, Alison

AU - Harrando, Ismail

AU - Laaksonen, Jorma

AU - Francis, Danny

AU - Troncy, Raphaël

AU - Laria Mantecon, Hector

N1 - | openaire: EC/H2020/780069/EU//MeMAD

PY - 2019/10/27

Y1 - 2019/10/27

N2 - This paper describes a multimodal approach proposed by the MeMAD team for the MediaEval 2019 “Predicting Media memorability” task. Our best approach is a weighted average method combining predictions made separately from visual and textual representations of videos. In particular, we augmented the provided textual descriptions with automatically generated deep captions. For long termmemorability, we obtained better scores using the short term predictions rather than the long term ones. Our best model achieves Spearman scores of 0.522 and 0.277 respectively for the short and long term predictions tasks.

AB - This paper describes a multimodal approach proposed by the MeMAD team for the MediaEval 2019 “Predicting Media memorability” task. Our best approach is a weighted average method combining predictions made separately from visual and textual representations of videos. In particular, we augmented the provided textual descriptions with automatically generated deep captions. For long termmemorability, we obtained better scores using the short term predictions rather than the long term ones. Our best model achieves Spearman scores of 0.522 and 0.277 respectively for the short and long term predictions tasks.

UR - http://ceur-ws.org/Vol-2670/

M3 - Chapter

T3 - CEUR Workshop Proceedings

BT - Working Notes Proceedings of the MediaEval 2019 Workshop, Sophia Antipolis, France, 27-30 October 2019

PB - CEUR

T2 - Multimedia Benchmark Workshop

Y2 - 27 October 2019 through 30 October 2019

ER -

Combining Textual and Visual Modeling for Predicting Media Memorability

Abstract

Publication series

Workshop

Access to Document

Other files and links

Fingerprint

MeMAD Laaksonen

Science-IT

Cite this

Combining Textual and Visual Modeling for Predicting Media Memorability

Abstract

Publication series

Workshop

Access to Document

Other files and links

Fingerprint

Projects

MeMAD Laaksonen

Equipment

Science-IT

Cite this