Projects per year
Abstract
Self-supervised speech models, such as the wav2vec2, have become extremely popular in the past few years. Their main appeal is that after their pre-training on a large amount of audio, they require only a small amount of supervised, finetuning data to achieve outstanding results. Despite their immense success, very little is understood about the pre-trained models and how finetuning changes them. In this work, we take the first steps towards a better understanding of wav2vec2 systems using model interpretation tools such as visualization and latent embedding clustering. Through our analysis, we gain new insights into the abilities of the pre-trained networks and the effect that finetuning has on them. We demonstrate that the clusters learned by the pre-trained model are just as important a factor as the supervised training data distribution in determining the accuracy of the finetuned system, which could aid us in selecting the most suitable pre-trained model for the supervised data.
Original language | English |
---|---|
Title of host publication | Proceedings of Interspeech 2023 |
Publisher | International Speech Communication Association (ISCA) |
Pages | 196-200 |
Number of pages | 5 |
DOIs | |
Publication status | Published - 20 Aug 2023 |
MoE publication type | A4 Conference publication |
Event | Interspeech - Dublin, Ireland Duration: 20 Aug 2023 → 24 Aug 2023 |
Publication series
Name | Interspeech |
---|---|
Publisher | International Speech Communication Association |
ISSN (Electronic) | 2958-1796 |
Conference
Conference | Interspeech |
---|---|
Country/Territory | Ireland |
City | Dublin |
Period | 20/08/2023 → 24/08/2023 |
Fingerprint
Dive into the research topics of 'Investigating wav2vec2 context representations and the effects of fine-tuning, a case-study of a Finnish model'. Together they form a unique fingerprint.Projects
- 1 Finished
-
USSEE: Understanding Speech and Scene with Ears and Eyes
Kurimo, M. (Principal investigator), Virkkunen, A. (Project Member) & Grósz, T. (Project Member)
01/01/2022 → 31/12/2024
Project: Academy of Finland: Other research funding