Multi-Task Learning for Jointly Detecting Depression and Emotion

Yazhou Zhang, Xiang Li*, Lu Rong, Prayag Tiwari

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingsScientificpeer-review

19 Citations (Scopus)
185 Downloads (Pure)

Abstract

Depression is a typical mood disease that makes people a persistent feeling of sadness and loss of interest and pleasure. Emotion thus comes into sight and is tightly entangled with depression in that one helps the understanding of the other. Depression and emotion detection has been a new research task. The central challenges in this task are multi-modal interaction and multi-task correlation. The existing approaches treat them as two separate tasks, and fail to model the relationships between them. In this paper, we propose an attentive multi-modal multitask learning framework, called AMM, to generically address such issues. The core modules are two attention mechanisms, viz. inter-modal (I {mathrm{e}}) and inter-task (I {t}) attentions. The main motivation of I {mathrm{e}} attention is to learn multi-modal fused representation. In contrast, It attention is proposed to learn the relationship between depression detection and emotion recognition. Extensive experiments are conducted on two large scale datasets, i.e., DAIC and multi-modal Getty Image depression (MGID). The results show the effectiveness of the proposed AMM framework, and also shows that AMM obtains better performance for the main task, i.e., depression detection with the help of the secondary emotion recognition task.

Original languageEnglish
Title of host publicationProceedings - 2021 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2021
EditorsYufei Huang, Lukasz Kurgan, Feng Luo, Xiaohua Tony Hu, Yidong Chen, Edward Dougherty, Andrzej Kloczkowski, Yaohang Li
PublisherIEEE
Pages3142-3149
Number of pages8
ISBN (Electronic)978-1-6654-0126-5
DOIs
Publication statusPublished - 2021
MoE publication typeA4 Conference publication
EventIEEE International Conference on Bioinformatics and Biomedicine - Virtual, Online, United States
Duration: 9 Dec 202112 Dec 2021

Conference

ConferenceIEEE International Conference on Bioinformatics and Biomedicine
Abbreviated titleBIBM
Country/TerritoryUnited States
CityVirtual, Online
Period09/12/202112/12/2021

Keywords

  • artificial intelligence
  • deep learning
  • emotion recognition
  • multi-modal depression detection
  • multi-task learning

Fingerprint

Dive into the research topics of 'Multi-Task Learning for Jointly Detecting Depression and Emotion'. Together they form a unique fingerprint.

Cite this