TY - JOUR
T1 - Comparing human and automatic speech recognition in a perceptual restoration experiment
AU - Remes, Ulpu
AU - Ramírez López, Ana
AU - Juvela, Lauri
AU - Palomäki, Kalle
AU - Brown, Guy J.
AU - Alku, Paavo
AU - Kurimo, Mikko
N1 - COIN
PY - 2016/7/11
Y1 - 2016/7/11
N2 - Speech that has been distorted by introducing spectral or temporal gaps is still perceived as continuous and complete by human listeners, so long as the gaps are filled with additive noise of sufficient intensity. When such perceptual restoration occurs, the speech is also more intelligible compared to the case in which noise has not been added in the gaps. This observation has motivated so-called 'missing data' systems for automatic speech recognition (ASR), but there have been few attempts to determine whether such systems are a good model of perceptual restoration in human listeners. Accordingly, the current paper evaluates missing data ASR in a perceptual restoration task. We evaluated two systems that use a new approach to bounded marginalisation in the cepstral domain, and a bounded conditional mean imputation method. Both methods model available speech information as a clean-speech posterior distribution that is subsequently passed to an ASR system. The proposed missing data ASR systems were evaluated using distorted speech, in which spectro-temporal gaps were optionally filled with additive noise. Speech recognition performance of the proposed systems was compared against a baseline ASR system, and with human speech recognition performance on the same task. We conclude that missing data methods improve speech recognition performance in a manner that is consistent with perceptual restoration in human listeners.
AB - Speech that has been distorted by introducing spectral or temporal gaps is still perceived as continuous and complete by human listeners, so long as the gaps are filled with additive noise of sufficient intensity. When such perceptual restoration occurs, the speech is also more intelligible compared to the case in which noise has not been added in the gaps. This observation has motivated so-called 'missing data' systems for automatic speech recognition (ASR), but there have been few attempts to determine whether such systems are a good model of perceptual restoration in human listeners. Accordingly, the current paper evaluates missing data ASR in a perceptual restoration task. We evaluated two systems that use a new approach to bounded marginalisation in the cepstral domain, and a bounded conditional mean imputation method. Both methods model available speech information as a clean-speech posterior distribution that is subsequently passed to an ASR system. The proposed missing data ASR systems were evaluated using distorted speech, in which spectro-temporal gaps were optionally filled with additive noise. Speech recognition performance of the proposed systems was compared against a baseline ASR system, and with human speech recognition performance on the same task. We conclude that missing data methods improve speech recognition performance in a manner that is consistent with perceptual restoration in human listeners.
KW - Automatic speech recognition
KW - Missing data
KW - Observation uncertainties
KW - Perceptual restoration
KW - Uncertainty propagation
UR - http://www.scopus.com/inward/record.url?scp=84936873584&partnerID=8YFLogxK
U2 - 10.1016/j.csl.2015.06.005
DO - 10.1016/j.csl.2015.06.005
M3 - Article
SN - 0885-2308
VL - 35
SP - 14
EP - 31
JO - Computer Speech and Language
JF - Computer Speech and Language
ER -