Projekteja vuodessa
Abstrakti
Unsupervised pretraining is an integral part of many natural language processing systems, and transfer learning with language models has achieved remarkable results in downstream tasks. In the clinical application of medical code assignment, diagnosis and procedure codes are inferred from lengthy clinical notes such as hospital discharge summaries. However, it is not clear if pretrained models are useful for medical code prediction without further architecture engineering. This paper conducts a comprehensive quantitative analysis of various contextualized language models' performances, pretrained in different domains, for medical code assignment from clinical notes. We propose a hierarchical fine-tuning architecture to capture interactions between distant words and adopt label-wise attention to exploit label information. Contrary to current trends, we demonstrate that a carefully trained classical CNN outperforms attention-based models on a MIMIC-III subset with frequent codes. Our empirical findings suggest directions for building robust medical code assignment models.
Alkuperäiskieli | Englanti |
---|---|
Artikkeli | 104998 |
Julkaisu | Computers in Biology and Medicine |
Vuosikerta | 139 |
Varhainen verkossa julkaisun päivämäärä | lokak. 2021 |
DOI - pysyväislinkit | |
Tila | Julkaistu - jouluk. 2021 |
OKM-julkaisutyyppi | A1 Alkuperäisartikkeli tieteellisessä aikakauslehdessä |
Sormenjälki
Sukella tutkimusaiheisiin 'Does the magic of BERT apply to medical code assignment? A quantitative study'. Ne muodostavat yhdessä ainutlaatuisen sormenjäljen.-
INTERVENE: International consortium for integrative genomics prediction
Kaski, S. (Vastuullinen tutkija)
01/01/2021 → 31/12/2025
Projekti: EU: Framework programmes funding
-
DATALIT: Data Literacy for Responsible Decision-Making
Marttinen, P. (Vastuullinen tutkija), Ji, S. (Projektin jäsen), Gröhn, T. (Projektin jäsen), Honkamaa, J. (Projektin jäsen), Kumar, Y. (Projektin jäsen), Pöllänen, A. (Projektin jäsen), Ojala, F. (Projektin jäsen), Raj, V. (Projektin jäsen) & Tiwari, P. (Projektin jäsen)
01/10/2020 → 30/09/2023
Projekti: Academy of Finland: Strategic research funding