A computational model of early language acquisition from audiovisual experiences of young infants

Okko Räsänen, Khazar Khorrami

Tutkimustuotos: Artikkeli kirjassa/konferenssijulkaisussaConference contributionScientificvertaisarvioitu

29 Lataukset (Pure)

Abstrakti

Earlier research has suggested that human infants might use statistical dependencies between speech and non-linguistic multimodal input to bootstrap their language learning before they know how to segment words from running speech. However, feasibility of this hypothesis in terms of real-world infant experiences has remained unclear. This paper presents a step towards a more realistic test of the multimodal bootstrapping hypothesis by describing a neural network model that can learn word segments and their meanings from referentially ambiguous acoustic input. The model is tested on recordings of real infant-caregiver interactions using utterance-level labels for concrete visual objects that were attended by the infant when caregiver spoke an utterance containing the name of the object, and using random visual labels for utterances during absence of attention. The results show that beginnings of lexical knowledge may indeed emerge from individually ambiguous learning scenarios. In addition, the hidden layers of the network show gradually increasing selectivity to phonetic categories as a function of layer depth, resembling models trained for phone recognition in a supervised manner.

AlkuperäiskieliEnglanti
OtsikkoProceedings of Interspeech
KustantajaInternational Speech Communication Association
Sivut3594-3598
Sivumäärä5
Vuosikerta2019-September
DOI - pysyväislinkit
TilaJulkaistu - 1 tammikuuta 2019
OKM-julkaisutyyppiA4 Artikkeli konferenssijulkaisuussa
TapahtumaInterspeech - Graz, Itävalta
Kesto: 15 syyskuuta 201919 syyskuuta 2019
https://www.interspeech2019.org/

Julkaisusarja

NimiInterspeech - Annual Conference of the International Speech Communication Association
ISSN (elektroninen)2308-457X

Conference

ConferenceInterspeech
MaaItävalta
KaupunkiGraz
Ajanjakso15/09/201919/09/2019
www-osoite

Sormenjälki Sukella tutkimusaiheisiin 'A computational model of early language acquisition from audiovisual experiences of young infants'. Ne muodostavat yhdessä ainutlaatuisen sormenjäljen.

  • Projektit

    • 2 Päättynyt

    Ihmisen ja koneen kielenoppimisen kontekstisidonnainen laskennallinen perusta

    Räsänen, O.

    31/12/201731/12/2017

    Projekti: Academy of Finland: Other research funding

    Ihmisen ja koneen kielenoppimisen kontekstisidonnainen laskennallinen perusta

    Räsänen, O.

    31/12/201731/12/2017

    Projekti: Academy of Finland: Other research funding

    Siteeraa tätä

    Räsänen, O., & Khorrami, K. (2019). A computational model of early language acquisition from audiovisual experiences of young infants. teoksessa Proceedings of Interspeech (Vuosikerta 2019-September, Sivut 3594-3598). (Interspeech - Annual Conference of the International Speech Communication Association). International Speech Communication Association. https://doi.org/10.21437/Interspeech.2019-1523