Skip to main navigation Skip to search Skip to main content

Classification errors distort findings in automated speech processing: Examples and solutions from child-development research

  • Lucas Gautheron*
  • , Evan Kidd
  • , Anton Malko
  • , Marvin Lavechin
  • , Alejandrina Cristia
  • *Corresponding author for this work

Research output: Contribution to journal › Article › peer-review

Abstract

With the advent of wearable recorders, scientists are increasingly turning to automated methods of analysis of audio and video data in order to measure children’s experience, behavior, and outcomes, with a sizable literature employing long-form audio-recordings to study language acquisition. While numerous articles report on the accuracy and reliability of the most popular automated classifiers, less has been written on the downstream effects of classification errors on measurements and statistical inferences (e.g., the estimate of correlations and effect sizes in regressions). This paper’s main contributions are drawing attention to downstream effects of confusion errors, and providing an approach to measure and potentially recover from these errors. Specifically, we use a Bayesian approach to study the effects of algorithmic errors on key scientific questions, including the effect of siblings on children’s language experience and the association between children’s production and their input. By fitting a joint model of speech behavior and algorithm behavior on real and simulated data, we show that classification errors can significantly distort estimates for both the most commonly used Language ENvironment Analysis (LENA™), and a slightly more accurate open-source alternative (the Voice Type Classifier from the ACLEW system). We further show that a Bayesian calibration approach for recovering unbiased estimates of effect sizes can be effective and insightful, but does not provide a foolproof solution.
Original languageEnglish
Article number168
Number of pages24
JournalBehavior Research Methods
Volume58
Issue number6
DOIs
Publication statusPublished - Jun 2026

Fingerprint

Dive into the research topics of 'Classification errors distort findings in automated speech processing: Examples and solutions from child-development research'. Together they form a unique fingerprint.

Cite this