Normal-to-Lombard adaptation of speech synthesis using long short-term memory recurrent neural networks

Research output: Contribution to journalArticleScientificpeer-review

Standard

Normal-to-Lombard adaptation of speech synthesis using long short-term memory recurrent neural networks. / Bollepalli, Bajibabu; Juvela, Lauri; Airaksinen, Manu; Valentini-Botinhao, Cassia; Alku, Paavo.

In: Speech Communication, Vol. 110, 01.07.2019, p. 64-75.

Research output: Contribution to journalArticleScientificpeer-review

Harvard

APA

Vancouver

Author

Bibtex - Download

@article{25aea363f4b74bf09bac8d5d3f3b04ab,
title = "Normal-to-Lombard adaptation of speech synthesis using long short-term memory recurrent neural networks",
abstract = "In this article, three adaptation methods are compared based on how well they change the speaking style of a neural network based text-to-speech (TTS) voice. The speaking style conversion adopted here is from normal to Lombard speech. The selected adaptation methods are: auxiliary features (AF), learning hidden unit contribution (LHUC), and fine-tuning (FT). Furthermore, four state-of-the-art TTS vocoders are compared in the same context. The evaluated vocoders are: GlottHMM, GlottDNN, STRAIGHT, and pulse model in log-domain (PML). Objective and subjective evaluations were conducted to study the performance of both the adaptation methods and the vocoders. In the subjective evaluations, speaking style similarity and speech intelligibility were assessed. In addition to acoustic model adaptation, phoneme durations were also adapted from normal to Lombard with the FT adaptation method. In objective evaluations and speaking style similarity tests, we found that the FT method outperformed the other two adaptation methods. In speech intelligibility tests, we found that there were no significant differences between vocoders although the PML vocoder showed slightly better performance compared to the three other vocoders.",
keywords = "Lombard, Auxiliary features, LHUC, Fine-tuning, LSTM, Adaptation, TTS",
author = "Bajibabu Bollepalli and Lauri Juvela and Manu Airaksinen and Cassia Valentini-Botinhao and Paavo Alku",
year = "2019",
month = "7",
day = "1",
doi = "10.1016/j.specom.2019.04.008",
language = "English",
volume = "110",
pages = "64--75",
journal = "Speech Communication",
issn = "0167-6393",
publisher = "Elsevier",

}

RIS - Download

TY - JOUR

T1 - Normal-to-Lombard adaptation of speech synthesis using long short-term memory recurrent neural networks

AU - Bollepalli, Bajibabu

AU - Juvela, Lauri

AU - Airaksinen, Manu

AU - Valentini-Botinhao, Cassia

AU - Alku, Paavo

PY - 2019/7/1

Y1 - 2019/7/1

N2 - In this article, three adaptation methods are compared based on how well they change the speaking style of a neural network based text-to-speech (TTS) voice. The speaking style conversion adopted here is from normal to Lombard speech. The selected adaptation methods are: auxiliary features (AF), learning hidden unit contribution (LHUC), and fine-tuning (FT). Furthermore, four state-of-the-art TTS vocoders are compared in the same context. The evaluated vocoders are: GlottHMM, GlottDNN, STRAIGHT, and pulse model in log-domain (PML). Objective and subjective evaluations were conducted to study the performance of both the adaptation methods and the vocoders. In the subjective evaluations, speaking style similarity and speech intelligibility were assessed. In addition to acoustic model adaptation, phoneme durations were also adapted from normal to Lombard with the FT adaptation method. In objective evaluations and speaking style similarity tests, we found that the FT method outperformed the other two adaptation methods. In speech intelligibility tests, we found that there were no significant differences between vocoders although the PML vocoder showed slightly better performance compared to the three other vocoders.

AB - In this article, three adaptation methods are compared based on how well they change the speaking style of a neural network based text-to-speech (TTS) voice. The speaking style conversion adopted here is from normal to Lombard speech. The selected adaptation methods are: auxiliary features (AF), learning hidden unit contribution (LHUC), and fine-tuning (FT). Furthermore, four state-of-the-art TTS vocoders are compared in the same context. The evaluated vocoders are: GlottHMM, GlottDNN, STRAIGHT, and pulse model in log-domain (PML). Objective and subjective evaluations were conducted to study the performance of both the adaptation methods and the vocoders. In the subjective evaluations, speaking style similarity and speech intelligibility were assessed. In addition to acoustic model adaptation, phoneme durations were also adapted from normal to Lombard with the FT adaptation method. In objective evaluations and speaking style similarity tests, we found that the FT method outperformed the other two adaptation methods. In speech intelligibility tests, we found that there were no significant differences between vocoders although the PML vocoder showed slightly better performance compared to the three other vocoders.

KW - Lombard

KW - Auxiliary features

KW - LHUC

KW - Fine-tuning

KW - LSTM

KW - Adaptation

KW - TTS

UR - http://www.scopus.com/inward/record.url?scp=85064711915&partnerID=8YFLogxK

U2 - 10.1016/j.specom.2019.04.008

DO - 10.1016/j.specom.2019.04.008

M3 - Article

VL - 110

SP - 64

EP - 75

JO - Speech Communication

JF - Speech Communication

SN - 0167-6393

ER -

ID: 33417581