Speech Waveform Synthesis from MFCC Sequences with Generative Adversarial Networks

Lauri Juvela; Bajibabu Bollepalli; Xin Wang; Hirokazu Kameoka; Manu Airaksinen; Junichi Yamagishi; Paavo Alku

doi:10.1109/ICASSP.2018.8461852

Speech Waveform Synthesis from MFCC Sequences with Generative Adversarial Networks

Lauri Juvela, Bajibabu Bollepalli, Xin Wang, Hirokazu Kameoka, Manu Airaksinen, Junichi Yamagishi, Paavo Alku

Speech Communication Technology

Tutkimustuotos: Artikkeli kirjassa/konferenssijulkaisussa › Conference article in proceedings › Scientific › vertaisarvioitu

50 Sitaatiot (Scopus)

340 Lataukset (Pure)

Abstrakti

This paper proposes a method for generating speech from filterbank mel frequency cepstral coefficients (MFCC), which are widely used in speech applications, such as ASR, but are generally considered unusable for speech synthesis. First, we predict fundamental frequency and voicing information from MFCCs with an autoregressive recurrent neural net. Second, the spectral envelope information contained in MFCCs is converted to all-pole filters, and a pitch-synchronous excitation model matched to these filters is trained. Finally, we introduce a generative adversarial network-based noise model to add a realistic high-frequency stochastic component to the modeled excitation signal. The results show that high quality speech reconstruction can be obtained, given only MFCC information at test time.

Alkuperäiskieli	Englanti
Otsikko	2018 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2018 - Proceedings
Julkaisupaikka	United States
Kustantaja	IEEE
Sivut	5679-5683
Sivumäärä	5
Vuosikerta	2018-April
ISBN (elektroninen)	978-1-5386-4658-8
ISBN (painettu)	978-1-5386-4659-5
DOI - pysyväislinkit	https://doi.org/10.1109/ICASSP.2018.8461852
Tila	Julkaistu - 10 syysk. 2018
OKM-julkaisutyyppi	A4 Artikkeli konferenssijulkaisussa
Tapahtuma	IEEE International Conference on Acoustics, Speech, and Signal Processing - Calgary, Kanada Kesto: 15 huhtik. 2018 → 20 huhtik. 2018 https://2018.ieeeicassp.org/

Julkaisusarja

Nimi	Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing
ISSN (elektroninen)	2379-190X

Conference

Conference	IEEE International Conference on Acoustics, Speech, and Signal Processing
Lyhennettä	ICASSP
Maa/Alue	Kanada
Kaupunki	Calgary
Ajanjakso	15/04/2018 → 20/04/2018
www-osoite	https://2018.ieeeicassp.org/

Pääsy asiakirjaan

10.1109/ICASSP.2018.8461852

ELEC-Juvela-et-al-IEEEE-ICASSP2018Accepted author manuscript, 273 KB

Muut tiedostot ja linkit

Link to publication in Scopus

Poikkitieteellinen parametrisen puhesynteesin tutkimusprojekti
Alku, P. (Vastuullinen tutkija), Bäckström, T. (Projektin jäsen), Juvela, L. (Projektin jäsen), Murtola, T. (Projektin jäsen), Nonavinakere Prabhakera, N. (Projektin jäsen), Bollepalli, B. (Projektin jäsen) & Airaksinen, M. (Projektin jäsen)
01/01/2018 → 31/12/2019
Projekti: Academy of Finland: Other research funding
Poikkitieteellinen parametrisen puhesynteesin tutkimusprojekti
Juvela, L. (Projektin jäsen), Bäckström, T. (Projektin jäsen), Pohjalainen, J. (Projektin jäsen), Gowda, D. (Projektin jäsen), Jokinen, E. (Projektin jäsen), Alku, P. (Vastuullinen tutkija), Bollepalli, B. (Projektin jäsen), Saeidi, R. (Projektin jäsen), Raitio, T. (Projektin jäsen), Kakouros, S. (Projektin jäsen) & Airaksinen, M. (Projektin jäsen)
01/01/2015 → 31/12/2017
Projekti: Academy of Finland: Other research funding

Science-IT
Hakala, M. (Manager)
Perustieteiden korkeakoulu
Laitteistot/tilat: Facility

Siteeraa tätä

Juvela, L., Bollepalli, B., Wang, X., Kameoka, H., Airaksinen, M., Yamagishi, J., & Alku, P. (2018). Speech Waveform Synthesis from MFCC Sequences with Generative Adversarial Networks. teoksessa 2018 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2018 - Proceedings (Vuosikerta 2018-April, Sivut 5679-5683). Artikkeli 8461852 (Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing). IEEE. https://doi.org/10.1109/ICASSP.2018.8461852

Juvela, Lauri ; Bollepalli, Bajibabu ; Wang, Xin et al. / Speech Waveform Synthesis from MFCC Sequences with Generative Adversarial Networks. 2018 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2018 - Proceedings. Vuosikerta 2018-April United States : IEEE, 2018. Sivut 5679-5683 (Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing).

@inproceedings{1d94f4f06d0b42b38d3f4851a6de8bf8,

title = "Speech Waveform Synthesis from MFCC Sequences with Generative Adversarial Networks",

abstract = "This paper proposes a method for generating speech from filterbank mel frequency cepstral coefficients (MFCC), which are widely used in speech applications, such as ASR, but are generally considered unusable for speech synthesis. First, we predict fundamental frequency and voicing information from MFCCs with an autoregressive recurrent neural net. Second, the spectral envelope information contained in MFCCs is converted to all-pole filters, and a pitch-synchronous excitation model matched to these filters is trained. Finally, we introduce a generative adversarial network-based noise model to add a realistic high-frequency stochastic component to the modeled excitation signal. The results show that high quality speech reconstruction can be obtained, given only MFCC information at test time.",

keywords = "Excitation modeling, Generative adversarial networks, Mel-filterbank inversion, MFCC, Pitch prediction",

author = "Lauri Juvela and Bajibabu Bollepalli and Xin Wang and Hirokazu Kameoka and Manu Airaksinen and Junichi Yamagishi and Paavo Alku",

year = "2018",

month = sep,

day = "10",

doi = "10.1109/ICASSP.2018.8461852",

language = "English",

isbn = "978-1-5386-4659-5",

volume = "2018-April",

series = "Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing",

publisher = "IEEE",

pages = "5679--5683",

booktitle = "2018 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2018 - Proceedings",

address = "United States",

note = "IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP ; Conference date: 15-04-2018 Through 20-04-2018",

url = "https://2018.ieeeicassp.org/",

}

Juvela, L, Bollepalli, B, Wang, X, Kameoka, H, Airaksinen, M, Yamagishi, J & Alku, P 2018, Speech Waveform Synthesis from MFCC Sequences with Generative Adversarial Networks. julkaisussa 2018 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2018 - Proceedings. Vuosikerta 2018-April, 8461852, Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, IEEE, United States, Sivut 5679-5683, IEEE International Conference on Acoustics, Speech, and Signal Processing, Calgary, Alberta, Kanada, 15/04/2018. https://doi.org/10.1109/ICASSP.2018.8461852

Speech Waveform Synthesis from MFCC Sequences with Generative Adversarial Networks. / Juvela, Lauri; Bollepalli, Bajibabu; Wang, Xin et al.
2018 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2018 - Proceedings. Vuosikerta 2018-April United States: IEEE, 2018. s. 5679-5683 8461852 (Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing).

Tutkimustuotos: Artikkeli kirjassa/konferenssijulkaisussa › Conference article in proceedings › Scientific › vertaisarvioitu

TY - GEN

T1 - Speech Waveform Synthesis from MFCC Sequences with Generative Adversarial Networks

AU - Juvela, Lauri

AU - Bollepalli, Bajibabu

AU - Wang, Xin

AU - Kameoka, Hirokazu

AU - Airaksinen, Manu

AU - Yamagishi, Junichi

AU - Alku, Paavo

PY - 2018/9/10

Y1 - 2018/9/10

N2 - This paper proposes a method for generating speech from filterbank mel frequency cepstral coefficients (MFCC), which are widely used in speech applications, such as ASR, but are generally considered unusable for speech synthesis. First, we predict fundamental frequency and voicing information from MFCCs with an autoregressive recurrent neural net. Second, the spectral envelope information contained in MFCCs is converted to all-pole filters, and a pitch-synchronous excitation model matched to these filters is trained. Finally, we introduce a generative adversarial network-based noise model to add a realistic high-frequency stochastic component to the modeled excitation signal. The results show that high quality speech reconstruction can be obtained, given only MFCC information at test time.

AB - This paper proposes a method for generating speech from filterbank mel frequency cepstral coefficients (MFCC), which are widely used in speech applications, such as ASR, but are generally considered unusable for speech synthesis. First, we predict fundamental frequency and voicing information from MFCCs with an autoregressive recurrent neural net. Second, the spectral envelope information contained in MFCCs is converted to all-pole filters, and a pitch-synchronous excitation model matched to these filters is trained. Finally, we introduce a generative adversarial network-based noise model to add a realistic high-frequency stochastic component to the modeled excitation signal. The results show that high quality speech reconstruction can be obtained, given only MFCC information at test time.

KW - Excitation modeling

KW - Generative adversarial networks

KW - Mel-filterbank inversion

KW - MFCC

KW - Pitch prediction

UR - http://www.scopus.com/inward/record.url?scp=85054230031&partnerID=8YFLogxK

U2 - 10.1109/ICASSP.2018.8461852

DO - 10.1109/ICASSP.2018.8461852

M3 - Conference article in proceedings

AN - SCOPUS:85054230031

SN - 978-1-5386-4659-5

VL - 2018-April

T3 - Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing

SP - 5679

EP - 5683

BT - 2018 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2018 - Proceedings

PB - IEEE

CY - United States

T2 - IEEE International Conference on Acoustics, Speech, and Signal Processing

Y2 - 15 April 2018 through 20 April 2018

ER -

Juvela L, Bollepalli B, Wang X, Kameoka H, Airaksinen M, Yamagishi J et al. Speech Waveform Synthesis from MFCC Sequences with Generative Adversarial Networks. julkaisussa 2018 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2018 - Proceedings. Vuosikerta 2018-April. United States: IEEE. 2018. s. 5679-5683. 8461852. (Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing). doi: 10.1109/ICASSP.2018.8461852

Speech Waveform Synthesis from MFCC Sequences with Generative Adversarial Networks

Abstrakti

Julkaisusarja

Conference

Pääsy asiakirjaan

Muut tiedostot ja linkit

Sormenjälki

Projektit

Poikkitieteellinen parametrisen puhesynteesin tutkimusprojekti

Poikkitieteellinen parametrisen puhesynteesin tutkimusprojekti

Laitteet

Science-IT

Siteeraa tätä