Exploring Ordinal Classification for Spoken Language Assessment

Anusha Porwal; Nhan Phan; Yaroslav Getman; Tamás Grósz; Mikko Kurimo

doi:10.21437/SLaTE.2025-32

Exploring Ordinal Classification for Spoken Language Assessment

Anusha Porwal^*
, Nhan Phan^*
, Yaroslav Getman
, Tamás Grósz
, Mikko Kurimo

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference article in proceedings › Scientific › peer-review

Abstract

Foreign language learning is a paramount skill in today's world, and the need for tools that can be used to practice it has risen. In this paper, we introduce our solutions for the 2025 Speak & Improve Challenge, specifically for the closed track of the Spoken Language Assessment task. In this track, only the provided training data and the pre-selected Language Model (BERT) could be used, so we focused on novel ways of training it. One major issue that we have identified is the choice of the learning objective; previous works on language assessment treated the problem as a regression or classification task, but neither is fitting for the provided unbalanced data. We hypothesize that the Ordinal Loss approach is more suitable, especially when combined with data augmentation. Our empirical results demonstrate that the Ordinal Loss is superior compared to the standard solutions, yielding .411 RMSE and .782 PCC, and data augmentation improves the model's robustness.

Original language	English
Title of host publication	Proceedings of 10th Workshop on Speech and Language Technology in Education (SLaTE)
Publisher	International Speech Communication Association (ISCA)
Pages	158-162
Number of pages	4
DOIs	https://doi.org/10.21437/SLaTE.2025-32
Publication status	Published - Aug 2025
MoE publication type	A4 Conference publication
Event	Workshop on Speech and Language Technology in Education - Nijmegen, Netherlands Duration: 22 Aug 2025 → 24 Aug 2025

Publication series

Name	ISCA International Workshop on Speech and Language Technology in Education
ISSN (Electronic)	2311-4975

Workshop

Workshop	Workshop on Speech and Language Technology in Education
Abbreviated title	SLaTE
Country/Territory	Netherlands
City	Nijmegen
Period	22/08/2025 → 24/08/2025

Funding

Research Council of Finland’s grants 322625, 345790, 355587 and 365233

Keywords

automatic language assessment
computer-assisted language learning
Ordinal classification

Access to Document

10.21437/SLaTE.2025-32

1 Finished
2 Active

DigiTala 2025: DigiTala in action - Automatic speaking assessment supporting integration
Kurimo, M. (Principal investigator), Raud, N. (Project Member), Phan, N. (Project Member), Getman, Y. (Project Member), Porwal, A. (Project Member) & Pauna, O.-P. (Project Member)
01/01/2025 → 31/12/2026
Project: RCF Other
AASIS: Automatic assessment of spoken interaction in second language
Kurimo, M. (Principal investigator), Getman, Y. (Project Member), Phan, N. (Project Member), Lun, T. (Project Member), Voskoboinik, E. (Project Member), Al-Ghezi, R. (Project Member) & Raud, N. (Project Member)
01/09/2023 → 31/08/2027
Project: RCF Academy Project
USSEE: Understanding speech and scene with ears and eyes (USSEE)
Laaksonen, J. (Principal investigator), Kainulainen, J. (Project Member), Saif, A. (Project Member), Wang, T.-J. (Project Member), Guo, Z. (Project Member), Arora, P. (Project Member), Riahi, I. (Project Member), Tiwari, H. (Project Member) & Pehlivan Tort, S. (Project Member)
01/01/2022 → 31/12/2024
Project: RCF Academy Project targeted call

Science-IT
Hakala, M. (Manager)
School of Science
Facility/equipment: Facility

Cite this

Porwal, A., Phan, N., Getman, Y., Grósz, T., & Kurimo, M. (2025). Exploring Ordinal Classification for Spoken Language Assessment. In Proceedings of 10th Workshop on Speech and Language Technology in Education (SLaTE) (pp. 158-162). (ISCA International Workshop on Speech and Language Technology in Education). International Speech Communication Association (ISCA). https://doi.org/10.21437/SLaTE.2025-32

@inproceedings{4ade6cc1dab14578b54eef412ad93b51,

title = "Exploring Ordinal Classification for Spoken Language Assessment",

abstract = "Foreign language learning is a paramount skill in today's world, and the need for tools that can be used to practice it has risen. In this paper, we introduce our solutions for the 2025 Speak \& Improve Challenge, specifically for the closed track of the Spoken Language Assessment task. In this track, only the provided training data and the pre-selected Language Model (BERT) could be used, so we focused on novel ways of training it. One major issue that we have identified is the choice of the learning objective; previous works on language assessment treated the problem as a regression or classification task, but neither is fitting for the provided unbalanced data. We hypothesize that the Ordinal Loss approach is more suitable, especially when combined with data augmentation. Our empirical results demonstrate that the Ordinal Loss is superior compared to the standard solutions, yielding .411 RMSE and .782 PCC, and data augmentation improves the model's robustness.",

keywords = "automatic language assessment, computer-assisted language learning, Ordinal classification",

author = "Anusha Porwal and Nhan Phan and Yaroslav Getman and Tam{\'a}s Gr{\'o}sz and Mikko Kurimo",

year = "2025",

month = aug,

doi = "10.21437/SLaTE.2025-32",

language = "English",

series = "ISCA International Workshop on Speech and Language Technology in Education",

publisher = "International Speech Communication Association (ISCA)",

pages = "158--162",

booktitle = "Proceedings of 10th Workshop on Speech and Language Technology in Education (SLaTE)",

address = "France",

note = "Workshop on Speech and Language Technology in Education, SLaTE ; Conference date: 22-08-2025 Through 24-08-2025",

}

Porwal, A, Phan, N , Getman, Y, Grósz, T & Kurimo, M 2025, Exploring Ordinal Classification for Spoken Language Assessment. in Proceedings of 10th Workshop on Speech and Language Technology in Education (SLaTE). ISCA International Workshop on Speech and Language Technology in Education, International Speech Communication Association (ISCA), pp. 158-162, Workshop on Speech and Language Technology in Education, Nijmegen, Netherlands, 22/08/2025. https://doi.org/10.21437/SLaTE.2025-32

Exploring Ordinal Classification for Spoken Language Assessment. / Porwal, Anusha; Phan, Nhan ; Getman, Yaroslav et al.
Proceedings of 10th Workshop on Speech and Language Technology in Education (SLaTE). International Speech Communication Association (ISCA), 2025. p. 158-162 (ISCA International Workshop on Speech and Language Technology in Education).

Research output: Chapter in Book/Report/Conference proceeding › Conference article in proceedings › Scientific › peer-review

TY - GEN

T1 - Exploring Ordinal Classification for Spoken Language Assessment

AU - Porwal, Anusha

AU - Phan, Nhan

AU - Getman, Yaroslav

AU - Grósz, Tamás

AU - Kurimo, Mikko

PY - 2025/8

Y1 - 2025/8

N2 - Foreign language learning is a paramount skill in today's world, and the need for tools that can be used to practice it has risen. In this paper, we introduce our solutions for the 2025 Speak & Improve Challenge, specifically for the closed track of the Spoken Language Assessment task. In this track, only the provided training data and the pre-selected Language Model (BERT) could be used, so we focused on novel ways of training it. One major issue that we have identified is the choice of the learning objective; previous works on language assessment treated the problem as a regression or classification task, but neither is fitting for the provided unbalanced data. We hypothesize that the Ordinal Loss approach is more suitable, especially when combined with data augmentation. Our empirical results demonstrate that the Ordinal Loss is superior compared to the standard solutions, yielding .411 RMSE and .782 PCC, and data augmentation improves the model's robustness.

AB - Foreign language learning is a paramount skill in today's world, and the need for tools that can be used to practice it has risen. In this paper, we introduce our solutions for the 2025 Speak & Improve Challenge, specifically for the closed track of the Spoken Language Assessment task. In this track, only the provided training data and the pre-selected Language Model (BERT) could be used, so we focused on novel ways of training it. One major issue that we have identified is the choice of the learning objective; previous works on language assessment treated the problem as a regression or classification task, but neither is fitting for the provided unbalanced data. We hypothesize that the Ordinal Loss approach is more suitable, especially when combined with data augmentation. Our empirical results demonstrate that the Ordinal Loss is superior compared to the standard solutions, yielding .411 RMSE and .782 PCC, and data augmentation improves the model's robustness.

KW - automatic language assessment

KW - computer-assisted language learning

KW - Ordinal classification

U2 - 10.21437/SLaTE.2025-32

DO - 10.21437/SLaTE.2025-32

M3 - Conference article in proceedings

T3 - ISCA International Workshop on Speech and Language Technology in Education

SP - 158

EP - 162

BT - Proceedings of 10th Workshop on Speech and Language Technology in Education (SLaTE)

PB - International Speech Communication Association (ISCA)

T2 - Workshop on Speech and Language Technology in Education

Y2 - 22 August 2025 through 24 August 2025

ER -

Porwal A, Phan N , Getman Y, Grósz T, Kurimo M. Exploring Ordinal Classification for Spoken Language Assessment. In Proceedings of 10th Workshop on Speech and Language Technology in Education (SLaTE). International Speech Communication Association (ISCA). 2025. p. 158-162. (ISCA International Workshop on Speech and Language Technology in Education). doi: 10.21437/SLaTE.2025-32

Exploring Ordinal Classification for Spoken Language Assessment

Abstract

Publication series

Workshop

Funding

Keywords

Access to Document

Fingerprint

Projects

DigiTala 2025: DigiTala in action - Automatic speaking assessment supporting integration

AASIS: Automatic assessment of spoken interaction in second language

USSEE: Understanding speech and scene with ears and eyes (USSEE)

Equipment

Science-IT

Cite this