Developing a high-stake digital spoken language proficiency assessment: Results from pilot tests

H. Kallio*, R. Hilden, M. Kurimo, M. Vainio, R. Karhila, Erik Lindroos

*Corresponding author for this work

Research output: Contribution to conferencePaperScientific

Abstract

The DigiTala project aims at developing a prototype of a technology-assisted high-stake spoken language proficiency assessment using multidisciplinary expertise from language education, signal processing and phonetics. The prototype will use automatic speech recognition (ASR) tool specially developed for educational second language (L2) use. The first target application in Finland will be the new digital matriculation examination, where language tests are recorded and digitized for assessments. The objectives are to enhance the reliability and practicality of oral language testing in a high-stakes context by clarifying the phonetic realizations of assessment criteria and developing methods that enable the utilization of ASR in the assessment process. A database of all recordings and assessments will provide a unique research opportunity for scrutinizing oral proficiency.

The study integrates modern technologies and principles of agile development with a traditional examination system to develop a stakeholder-friendly assessment. Students and their teachers from five secondary schools in Finland have participated in the study. A group of ten experts on the target language will assess the fluency and pronunciation of speech samples from more than 200 students. The assessment criteria is based on the Common European Framework of Reference. Automatic assessments will be applied when the ASR tool is trained for L2 purposes.

This article will present the assessment application developed through the one-year project and the results from questionnaires collected from secondary school students and teachers who participated in pilot tests. The article will also report findings on the reliabilities on teacher and automatic assessments. The results will provide important information about the benefits as well as obstacles and challenges of a digitized assessment system.

Original languageEnglish
Pages1214-1214
Number of pages1
DOIs
Publication statusPublished - 2016
MoE publication typeNot Eligible
EventInternational Technology, Education and Development Conference - Valencia, Spain
Duration: 7 Mar 20169 Mar 2016
Conference number: 10

Conference

ConferenceInternational Technology, Education and Development Conference
Abbreviated titleINTED
CountrySpain
CityValencia
Period07/03/201609/03/2016

Keywords

  • Second language
  • assessment
  • ASR
  • technology

Cite this