Cycle-consistent adversarial networks for non-parallel vocal effort based speaking style conversion

Shreyas Seshadri; Lauri Juvela; Junichi Yamagishi; Okko Räsänen; Paavo Alku

doi:10.1109/ICASSP.2019.8682648

Cycle-consistent adversarial networks for non-parallel vocal effort based speaking style conversion

Shreyas Seshadri, Lauri Juvela, Junichi Yamagishi, Okko Räsänen, Paavo Alku

Speech Communication Technology

Tutkimustuotos: Artikkeli kirjassa/konferenssijulkaisussa › Conference article in proceedings › Scientific › vertaisarvioitu

13 Sitaatiot (Scopus)

222 Lataukset (Pure)

Abstrakti

Speaking style conversion (SSC) is the technology of converting natural speech signals from one style to another. In this study, we propose the use of cycle-consistent adversarial networks (CycleGANs) for converting styles with varying vocal effort, and focus on conversion between normal and Lombard styles as a case study of this problem. We propose a parametric approach that uses the Pulse Model in Log domain (PML) vocoder to extract speech features. These features are mapped using the CycleGAN from utterances in the source style to the corresponding features of target speech. Finally, the mapped features are converted to a Lombard speech waveform with the PML. The CycleGAN was compared in subjective listening tests with 2 other standard mapping methods used in conversion, and the CycleGAN was found to have the best performance in terms of speech quality and in terms of the magnitude of the perceptual change between the two styles.

Alkuperäiskieli	Englanti
Otsikko	ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
Kustantaja	IEEE
Sivut	6835 - 6839
Sivumäärä	5
ISBN (elektroninen)	978-1-4799-8131-1
ISBN (painettu)	978-1-4799-8132-8
DOI - pysyväislinkit	https://doi.org/10.1109/ICASSP.2019.8682648
Tila	Julkaistu - 1 toukok. 2019
OKM-julkaisutyyppi	A4 Artikkeli konferenssijulkaisussa
Tapahtuma	IEEE International Conference on Acoustics, Speech, and Signal Processing - Brighton, Iso-Britannia Kesto: 12 toukok. 2019 → 17 toukok. 2019 Konferenssinumero: 44

Julkaisusarja

Nimi	Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing
ISSN (painettu)	1520-6149
ISSN (elektroninen)	2379-190X

Conference

Conference	IEEE International Conference on Acoustics, Speech, and Signal Processing
Lyhennettä	ICASSP
Maa/Alue	Iso-Britannia
Kaupunki	Brighton
Ajanjakso	12/05/2019 → 17/05/2019

Pääsy asiakirjaan

10.1109/ICASSP.2019.8682648

ELEC_Seshadri_Cycle_consistent_vocal_effort_ICASSP19Accepted author manuscript, 746 KB

Muut tiedostot ja linkit

Link to publication in Scopus

Poikkitieteellinen parametrisen puhesynteesin tutkimusprojekti
Alku, P. (Vastuullinen tutkija), Bäckström, T. (Projektin jäsen), Juvela, L. (Projektin jäsen), Murtola, T. (Projektin jäsen), Nonavinakere Prabhakera, N. (Projektin jäsen), Bollepalli, B. (Projektin jäsen) & Airaksinen, M. (Projektin jäsen)
01/01/2018 → 31/12/2019
Projekti: Academy of Finland: Other research funding
-: Ihmisen ja koneen kielenoppimisen kontekstisidonnainen laskennallinen perusta
Räsänen, O.
31/12/2017 → 31/08/2023
Projekti: Academy of Finland: Other research funding
ACLEW: Lasten kielikokemuksien kartoitus ja analyysi koko maailman mittakaavassa
Räsänen, O. (Vastuullinen tutkija) & Seshadri, S. (Projektin jäsen)
01/06/2017 → 31/05/2020
Projekti: Academy of Finland: Other research funding

Siteeraa tätä

Seshadri, S., Juvela, L., Yamagishi, J., Räsänen, O., & Alku, P. (2019). Cycle-consistent adversarial networks for non-parallel vocal effort based speaking style conversion. teoksessa ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (Sivut 6835 - 6839). Artikkeli 8682648 (Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing). IEEE. https://doi.org/10.1109/ICASSP.2019.8682648

Seshadri, Shreyas ; Juvela, Lauri ; Yamagishi, Junichi et al. / Cycle-consistent adversarial networks for non-parallel vocal effort based speaking style conversion. ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2019. Sivut 6835 - 6839 (Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing).

@inproceedings{7b747ee9e78c4a67bc52dbcf77767c88,

title = "Cycle-consistent adversarial networks for non-parallel vocal effort based speaking style conversion",

abstract = "Speaking style conversion (SSC) is the technology of converting natural speech signals from one style to another. In this study, we propose the use of cycle-consistent adversarial networks (CycleGANs) for converting styles with varying vocal effort, and focus on conversion between normal and Lombard styles as a case study of this problem. We propose a parametric approach that uses the Pulse Model in Log domain (PML) vocoder to extract speech features. These features are mapped using the CycleGAN from utterances in the source style to the corresponding features of target speech. Finally, the mapped features are converted to a Lombard speech waveform with the PML. The CycleGAN was compared in subjective listening tests with 2 other standard mapping methods used in conversion, and the CycleGAN was found to have the best performance in terms of speech quality and in terms of the magnitude of the perceptual change between the two styles.",

author = "Shreyas Seshadri and Lauri Juvela and Junichi Yamagishi and Okko R{\"a}s{\"a}nen and Paavo Alku",

year = "2019",

month = may,

day = "1",

doi = "10.1109/ICASSP.2019.8682648",

language = "English",

isbn = "978-1-4799-8132-8",

series = "Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing",

publisher = "IEEE",

pages = "6835 -- 6839",

booktitle = "ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)",

address = "United States",

note = "IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP ; Conference date: 12-05-2019 Through 17-05-2019",

}

Seshadri, S, Juvela, L, Yamagishi, J, Räsänen, O & Alku, P 2019, Cycle-consistent adversarial networks for non-parallel vocal effort based speaking style conversion. julkaisussa ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)., 8682648, Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, IEEE, Sivut 6835 - 6839, IEEE International Conference on Acoustics, Speech, and Signal Processing, Brighton, Iso-Britannia, 12/05/2019. https://doi.org/10.1109/ICASSP.2019.8682648

Cycle-consistent adversarial networks for non-parallel vocal effort based speaking style conversion. / Seshadri, Shreyas; Juvela, Lauri; Yamagishi, Junichi et al.
ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2019. s. 6835 - 6839 8682648 (Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing).

Tutkimustuotos: Artikkeli kirjassa/konferenssijulkaisussa › Conference article in proceedings › Scientific › vertaisarvioitu

TY - GEN

T1 - Cycle-consistent adversarial networks for non-parallel vocal effort based speaking style conversion

AU - Seshadri, Shreyas

AU - Juvela, Lauri

AU - Yamagishi, Junichi

AU - Räsänen, Okko

AU - Alku, Paavo

N1 - Conference code: 44

PY - 2019/5/1

Y1 - 2019/5/1

N2 - Speaking style conversion (SSC) is the technology of converting natural speech signals from one style to another. In this study, we propose the use of cycle-consistent adversarial networks (CycleGANs) for converting styles with varying vocal effort, and focus on conversion between normal and Lombard styles as a case study of this problem. We propose a parametric approach that uses the Pulse Model in Log domain (PML) vocoder to extract speech features. These features are mapped using the CycleGAN from utterances in the source style to the corresponding features of target speech. Finally, the mapped features are converted to a Lombard speech waveform with the PML. The CycleGAN was compared in subjective listening tests with 2 other standard mapping methods used in conversion, and the CycleGAN was found to have the best performance in terms of speech quality and in terms of the magnitude of the perceptual change between the two styles.

AB - Speaking style conversion (SSC) is the technology of converting natural speech signals from one style to another. In this study, we propose the use of cycle-consistent adversarial networks (CycleGANs) for converting styles with varying vocal effort, and focus on conversion between normal and Lombard styles as a case study of this problem. We propose a parametric approach that uses the Pulse Model in Log domain (PML) vocoder to extract speech features. These features are mapped using the CycleGAN from utterances in the source style to the corresponding features of target speech. Finally, the mapped features are converted to a Lombard speech waveform with the PML. The CycleGAN was compared in subjective listening tests with 2 other standard mapping methods used in conversion, and the CycleGAN was found to have the best performance in terms of speech quality and in terms of the magnitude of the perceptual change between the two styles.

UR - http://www.scopus.com/inward/record.url?scp=85069001234&partnerID=8YFLogxK

U2 - 10.1109/ICASSP.2019.8682648

DO - 10.1109/ICASSP.2019.8682648

M3 - Conference article in proceedings

SN - 978-1-4799-8132-8

T3 - Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing

SP - 6835

EP - 6839

BT - ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

PB - IEEE

T2 - IEEE International Conference on Acoustics, Speech, and Signal Processing

Y2 - 12 May 2019 through 17 May 2019

ER -

Seshadri S, Juvela L, Yamagishi J, Räsänen O, Alku P. Cycle-consistent adversarial networks for non-parallel vocal effort based speaking style conversion. julkaisussa ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE. 2019. s. 6835 - 6839. 8682648. (Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing). doi: 10.1109/ICASSP.2019.8682648

Cycle-consistent adversarial networks for non-parallel vocal effort based speaking style conversion

Abstrakti

Julkaisusarja

Conference

Pääsy asiakirjaan

Muut tiedostot ja linkit

Sormenjälki

Projektit

Poikkitieteellinen parametrisen puhesynteesin tutkimusprojekti

-: Ihmisen ja koneen kielenoppimisen kontekstisidonnainen laskennallinen perusta

ACLEW: Lasten kielikokemuksien kartoitus ja analyysi koko maailman mittakaavassa

Siteeraa tätä