Projekteja vuodessa
Abstrakti
We introduce the CaptainA mobile app, designed to meet
the needs of second language (L2) learners engaged in self-
study of Finnish, with potential applicability to other languages.
Our app can provide automatic speaking assessment (ASA) of
task completion in picture-based tasks, along with grading ex-
planations and corrective feedback. It can also automatically
generate pictures for visual tasks, providing users with unlim-
ited practice opportunities. The mobile app is based on our
framework that combines visual natural language generation
(NLG), automatic speech recognition (ASR), and prompting
large language model (LLM) for low-resource language. Our
goal is to promote the development of next-generation speech-
based computer-assisted language learning (CALL) systems ca-
pable of providing automatic scoring with feedback for learn-
ers, even when minimal speech data of L2 learners is available.
While the mobile app demonstration is designed for Finnish, the
app can also be tested in English.
the needs of second language (L2) learners engaged in self-
study of Finnish, with potential applicability to other languages.
Our app can provide automatic speaking assessment (ASA) of
task completion in picture-based tasks, along with grading ex-
planations and corrective feedback. It can also automatically
generate pictures for visual tasks, providing users with unlim-
ited practice opportunities. The mobile app is based on our
framework that combines visual natural language generation
(NLG), automatic speech recognition (ASR), and prompting
large language model (LLM) for low-resource language. Our
goal is to promote the development of next-generation speech-
based computer-assisted language learning (CALL) systems ca-
pable of providing automatic scoring with feedback for learn-
ers, even when minimal speech data of L2 learners is available.
While the mobile app demonstration is designed for Finnish, the
app can also be tested in English.
| Alkuperäiskieli | Englanti |
|---|---|
| Otsikko | Interspeech 2024 |
| Kustantaja | International Society for Computers and Their Applications (ISCA) |
| Sivut | 5212-5213 |
| Sivumäärä | 2 |
| Tila | Julkaistu - 1 syysk. 2024 |
| OKM-julkaisutyyppi | A4 Artikkeli konferenssijulkaisussa |
| Tapahtuma | Interspeech - Kos Island, Kreikka Kesto: 1 syysk. 2024 → 5 syysk. 2024 |
Julkaisusarja
| Nimi | Interspeech |
|---|---|
| ISSN (elektroninen) | 2308-457X |
Conference
| Conference | Interspeech |
|---|---|
| Maa/Alue | Kreikka |
| Kaupunki | Kos Island |
| Ajanjakso | 01/09/2024 → 05/09/2024 |
Sormenjälki
Sukella tutkimusaiheisiin 'CaptainA self-study mobile app for practising speaking: task completion assessment and feedback with generative AI'. Ne muodostavat yhdessä ainutlaatuisen sormenjäljen.-
AASIS: Automatic assessment of spoken interaction in second language
Kurimo, M. (Vastuullinen tutkija), Getman, Y. (Projektin jäsen), Voskoboinik, E. (Projektin jäsen), Phan, N. (Projektin jäsen) & Al-Ghezi, R. (Projektin jäsen)
01/09/2023 → 31/08/2027
Projekti: Academy of Finland: Other research funding
-
TEFLON: Technology-enhanced foreign and second-language learning of Nordic languages
Kurimo, M. (Vastuullinen tutkija), Smolander, A.-R. (Projektin jäsen), Getman, Y. (Projektin jäsen), Phan, N. (Projektin jäsen), Grósz, T. (Projektin jäsen), Voskoboinik, E. (Projektin jäsen) & Al-Ghezi, R. (Projektin jäsen)
01/04/2021 → 31/03/2025
Projekti: Other external funding: Other foreign funding
-
DigiTala: Aka-Digi Tala
Kurimo, M. (Vastuullinen tutkija), Getman, Y. (Projektin jäsen), Voskoboinik, E. (Projektin jäsen) & Al-Ghezi, R. (Projektin jäsen)
01/01/2020 → 31/08/2023
Projekti: Academy of Finland: Other research funding