Listening like a speech-training app: Expert and non-expert listeners’ goodness ratings of children’s speech

Sofia Strömbergsson*, Molly Fröjdh, Magdalena Pettersson, Tamás Grósz, Yaroslav Getman, Mikko Kurimo

*Corresponding author for this work

Research output: Contribution to journalArticleScientificpeer-review

21 Downloads (Pure)

Abstract

Speech training apps are being developed that provide automatic feedback concerning children’s production of known target words, as a score on a 1–5 scale. However, this ‘goodness’ scale is still poorly understood. We investigated listeners’ ratings of ‘how many stars the app should provide as feedback’ on children’s utterances, and whether listener agreement is affected by clinical experience and/or access to anchor stimuli. In addition, we explored the association between goodness ratings and clinical measures of speech accuracy; the Percentage of Consonants Correct (PCC) and the Percentage of Phonemes Correct (PPC). Twenty speech-language pathologists and 20 non-expert listeners participated; half of the listeners in each group had access to anchor stimuli. The listeners rated 120 words, collected from children with and without speech sound disorder. Concerning reliability, intra-rater agreement was generally high, whereas inter-rater agreement was moderate. Access to anchor stimuli was associated with higher agreement, but only for non-expert listeners. Concerning the association between goodness ratings and the PCC/PPC, correlations were moderate for both listener groups, under both conditions. The results indicate that the task of rating goodness is difficult, regardless of clinical experience, and that access to anchor stimuli is insufficient for achieving reliable ratings. This raises concerns regarding the 1–5 rating scale as the means of feedback in speech training apps. More specific listener instructions, particularly regarding the intended context for the app, are suggested in collection of human ratings underlying the development of speech training apps. Until then, alternative means of feedback should be preferred.

Original languageEnglish
Number of pages23
JournalClinical Linguistics and Phonetics
Volume38
Issue number4
DOIs
Publication statusPublished - 9 Jun 2024
MoE publication typeA1 Journal article-refereed

Keywords

  • automatic assessment
  • perceptual assessment
  • Speech accuracy
  • speech sound disorder

Fingerprint

Dive into the research topics of 'Listening like a speech-training app: Expert and non-expert listeners’ goodness ratings of children’s speech'. Together they form a unique fingerprint.

Cite this