Approaching human performance in noise robust automatic speech recognition 

Modern automatic speech recognition systems are able to achieve human-like performance on read speech in relatively noise-free environments. However, in the presence of heavily deteriorating noise, the gap between human and machine recognition remains large. The work presented in the thesis is aimed to enhance the speech recognition performance in varying noise and low signal-to-noise ratio conditions by improving the short-time spectral analysis of the speech signal and the spectrographic mask estimation in the missing data framework. In the thesis, the fast Fourier transformation based spectrum estimation of Mel-frequency cepstral coefficients is substituted with extended weighted linear prediction. Temporal weighting in linear predictive analysis emphasizes the high amplitude samples that are assumed less corrupted by noise and attenuates the others. Extending the weighting to separately apply to each lag in the prediction of each sample arguably offers more modeling power for deteriorated speech. The extended weighted linear prediction is shown to exceed the recognition performance of conventional linear prediction, weighted linear prediction and fast Fourier transformation based feature extraction. Missing data methods assume that only part of the spectro-temporal components of the deteriorated signal are corrupted by noise while the speech-dominant components hold the reliable information that can be used in recognition. Two spectrographic mask estimation techniques based on binary classification of features are proposed in the thesis. The first method is founded on a comprehensive set of design features and the second on the Gaussian-Bernoulli restricted Boltzmann machine that learns the feature set automatically. Both mask estimation methods are shown to outperform their respective reference mask estimation methods in recognition accuracy. All the proposed noise robust techniques are immediately applicable to automatic speech recognition. With further refinement, the mask estimation methods could also be applied to hearing aids since they are able to attenuate the background noise thus increasing the speech intelligibility.