Reproducibility analysis of automated deep learning based localisation of mandibular canals on a temporal CBCT dataset

Jorma Järnstedt*, Jaakko Sahlsten, Joel Jaskari, Kimmo Kaski*, Helena Mehtonen, Ari Hietanen, Osku Sundqvist, Vesa Varjonen, Vesa Mattila, Sangsom Prapayasatok, Sakarat Nalampang

*Corresponding author for this work

Research output: Contribution to journalArticleScientificpeer-review

7 Downloads (Pure)


Preoperative radiological identification of mandibular canals is essential for maxillofacial surgery. This study demonstrates the reproducibility of a deep learning system (DLS) by evaluating its localisation performance on 165 heterogeneous cone beam computed tomography (CBCT) scans from 72 patients in comparison to an experienced radiologist’s annotations. We evaluated the performance of the DLS using the symmetric mean curve distance (SMCD), the average symmetric surface distance (ASSD), and the Dice similarity coefficient (DSC). The reproducibility of the SMCD was assessed using the within-subject coefficient of repeatability (RC). Three other experts rated the diagnostic validity twice using a 0–4 Likert scale. The reproducibility of the Likert scoring was assessed using the repeatability measure (RM). The RC of SMCD was 0.969 mm, the median (interquartile range) SMCD and ASSD were 0.643 (0.186) mm and 0.351 (0.135) mm, respectively, and the mean (standard deviation) DSC was 0.548 (0.138). The DLS performance was most affected by postoperative changes. The RM of the Likert scoring was 0.923 for the radiologist and 0.877 for the DLS. The mean (standard deviation) Likert score was 3.94 (0.27) for the radiologist and 3.84 (0.65) for the DLS. The DLS demonstrated proficient qualitative and quantitative reproducibility, temporal generalisability, and clinical validity.

Original languageEnglish
Article number14159
JournalScientific Reports
Issue number1
Publication statusPublished - Dec 2023
MoE publication typeA1 Journal article-refereed


Dive into the research topics of 'Reproducibility analysis of automated deep learning based localisation of mandibular canals on a temporal CBCT dataset'. Together they form a unique fingerprint.

Cite this