Transferring Neural Speech Waveform Synthesizers to Musical Instrument Sounds Generation

Yi Zhao, Xin Wang, Lauri Juvela, Junichi Yamagishi

Tutkimustuotos: Artikkeli kirjassa/konferenssijulkaisussaConference article in proceedingsScientificvertaisarvioitu

8 Sitaatiot (Scopus)

Abstrakti

Recent neural waveform synthesizers such as WaveNet, WaveG-low, and the neural-source-filter (NSF) model have shown good performance in speech synthesis despite their different methods of waveform generation. The similarity between speech and music audio synthesis techniques suggests interesting avenues to explore in terms of the best way to apply speech synthesizers in the music domain. This work compares three neural synthesizers used for musical instrument sounds generation under three scenarios: training from scratch on music data, zero-shot learning from the speech domain, and fine-tuning-based adaptation from the speech to the music domain. The results of a large-scale perceptual test demonstrated that the performance of three synthesizers improved when they were pre-trained on speech data and fine-tuned on music data, which indicates the usefulness of knowledge from speech data for music audio generation. Among the synthesizers, WaveGlow showed the best potential in zero-shot learning while NSF performed best in the other scenarios and could generate samples that were perceptually close to natural audio.

AlkuperäiskieliEnglanti
Otsikko2020 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2020 - Proceedings
KustantajaIEEE
Sivut6269-6273
Sivumäärä5
ISBN (elektroninen)9781509066315
DOI - pysyväislinkit
TilaJulkaistu - toukok. 2020
OKM-julkaisutyyppiA4 Artikkeli konferenssijulkaisussa
TapahtumaIEEE International Conference on Acoustics, Speech, and Signal Processing - Virtual conference, Barcelona, Espanja
Kesto: 4 toukok. 20208 toukok. 2020
Konferenssinumero: 45

Julkaisusarja

NimiProceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing
ISSN (painettu)1520-6149
ISSN (elektroninen)2379-190X

Conference

ConferenceIEEE International Conference on Acoustics, Speech, and Signal Processing
LyhennettäICASSP
Maa/AlueEspanja
KaupunkiBarcelona
Ajanjakso04/05/202008/05/2020
MuuVirtual conference

Sormenjälki

Sukella tutkimusaiheisiin 'Transferring Neural Speech Waveform Synthesizers to Musical Instrument Sounds Generation'. Ne muodostavat yhdessä ainutlaatuisen sormenjäljen.

Siteeraa tätä