Discovering Relevant Sub-spaces of BERT, Wav2Vec 2.0, ELECTRA and ViT Embeddings for Humor and Mimicked Emotion Recognition with Integrated Gradients

Tutkimustuotos: Artikkeli kirjassa/konferenssijulkaisussaConference article in proceedingsScientificvertaisarvioitu

1 Sitaatiot (Scopus)
35 Lataukset (Pure)

Abstrakti

Large-scale, pre-trained models revolutionized the field of sentiment analysis and enabled multimodal systems to be quickly developed. In this paper, we address two challenges posed by the Multimodal Sentiment Analysis (MuSe) 2023 competition by focusing on automatically detecting cross-cultural humor and predicting three continuous emotion targets from user-generated videos. Multiple methods in the literature already demonstrate the importance of embedded features generated by popular pre-trained neural solutions. Based on their success, we can assume that the embedded space consists of several sub-spaces relevant to different tasks. Our aim is to automatically identify the task-specific sub-spaces of various embeddings by interpreting the baseline neural models. Once the relevant dimensions are located, we train a new model using only those features, which leads to similar or slightly better results with a considerably smaller and faster model. The best Humor Detection model using only the relevant sub-space of audio embeddings contained approximately 54% fewer parameters than the one processing the whole encoded vector, required 48% less time to be trained and even outperformed the larger model. Our empirical results validate that, indeed, only a portion of the embedding space is needed to achieve good performance. Our solution could be considered a novel form of knowledge distillation, which enables new ways of transferring knowledge from one model into another.
AlkuperäiskieliEnglanti
OtsikkoMuSe '23: Proceedings of the 4th on Multimodal Sentiment Analysis Challenge and Workshop: Mimicked Emotions, Humour and Personalisation
KustantajaACM
Sivut27-34
Sivumäärä8
ISBN (painettu)979-8-4007-0270-9
DOI - pysyväislinkit
TilaJulkaistu - 2 marrask. 2023
OKM-julkaisutyyppiA4 Artikkeli konferenssijulkaisussa
TapahtumaACM International Conference on Multimedia - Ottawa, Kanada
Kesto: 29 lokak. 202329 lokak. 2023
Konferenssinumero: 31

Conference

ConferenceACM International Conference on Multimedia
LyhennettäMM
Maa/AlueKanada
KaupunkiOttawa
Ajanjakso29/10/202329/10/2023

Sormenjälki

Sukella tutkimusaiheisiin 'Discovering Relevant Sub-spaces of BERT, Wav2Vec 2.0, ELECTRA and ViT Embeddings for Humor and Mimicked Emotion Recognition with Integrated Gradients'. Ne muodostavat yhdessä ainutlaatuisen sormenjäljen.

Siteeraa tätä