A thorough evaluation of the Language Environment Analysis (LENA) system

Alejandrina Cristia*, Marvin Lavechin, Camila Scaff, Melanie Soderstrom, Caroline Rowland, Okko Räsänen, John Bunce, Elika Bergelson

*Corresponding author for this work

Research output: Contribution to journalArticleScientificpeer-review

13 Citations (Scopus)
3 Downloads (Pure)

Abstract

In the previous decade, dozens of studies involving thousands of children across several research disciplines have made use of a combined daylong audio-recorder and automated algorithmic analysis called the LENA system, which aims to assess children’s language environment. While the system’s prevalence in the language acquisition domain is steadily growing, there are only scattered validation efforts on only some of its key characteristics. Here, we assess the LENA system’s accuracy across all of its key measures: speaker classification, Child Vocalization Counts (CVC), Conversational Turn Counts (CTC), and Adult Word Counts (AWC). Our assessment is based on manual annotation of clips that have been randomly or periodically sampled out of daylong recordings, collected from (a) populations similar to the system’s original training data (North American English-learning children aged 3-36 months), (b) children learning another dialect of English (UK), and (c) slightly older children growing up in a different linguistic and socio-cultural setting (Tsimane’ learners in rural Bolivia). We find reasonably high accuracy in some measures (AWC, CVC), with more problematic levels of performance in others (CTC, precision of male adults and other children). Statistical analyses do not support the view that performance is worse for children who are dissimilar from the LENA original training set. Whether LENA results are accurate enough for a given research, educational, or clinical application depends largely on the specifics at hand. We therefore conclude with a set of recommendations to help researchers make this determination for their goals.

Original languageEnglish
Number of pages20
JournalBehavior Research Methods
Early online date29 Jul 2020
DOIs
Publication statusPublished - 29 Jul 2020
MoE publication typeA1 Journal article-refereed

Keywords

  • Adult Word Count
  • Agreement
  • Child Vocalization Count
  • Conversational Turn Count
  • English
  • Human transcription
  • LENA
  • Measurement error
  • Method comparison
  • Reliability
  • Speech technology
  • Tsimane’

Fingerprint

Dive into the research topics of 'A thorough evaluation of the Language Environment Analysis (LENA) system'. Together they form a unique fingerprint.

Cite this