A computational model of early language acquisition from audiovisual experiences of young infants

Okko Räsänen, Khazar Khorrami

    Research output: Chapter in Book/Report/Conference proceedingConference contributionScientificpeer-review

    5 Citations (Scopus)
    91 Downloads (Pure)

    Abstract

    Earlier research has suggested that human infants might use statistical dependencies between speech and non-linguistic multimodal input to bootstrap their language learning before they know how to segment words from running speech. However, feasibility of this hypothesis in terms of real-world infant experiences has remained unclear. This paper presents a step towards a more realistic test of the multimodal bootstrapping hypothesis by describing a neural network model that can learn word segments and their meanings from referentially ambiguous acoustic input. The model is tested on recordings of real infant-caregiver interactions using utterance-level labels for concrete visual objects that were attended by the infant when caregiver spoke an utterance containing the name of the object, and using random visual labels for utterances during absence of attention. The results show that beginnings of lexical knowledge may indeed emerge from individually ambiguous learning scenarios. In addition, the hidden layers of the network show gradually increasing selectivity to phonetic categories as a function of layer depth, resembling models trained for phone recognition in a supervised manner.

    Original languageEnglish
    Title of host publicationProceedings of Interspeech
    PublisherInternational Speech Communication Association
    Pages3594-3598
    Number of pages5
    Volume2019-September
    DOIs
    Publication statusPublished - 1 Jan 2019
    MoE publication typeA4 Article in a conference publication
    EventInterspeech - Graz, Austria
    Duration: 15 Sep 201919 Sep 2019
    https://www.interspeech2019.org/

    Publication series

    NameInterspeech - Annual Conference of the International Speech Communication Association
    ISSN (Electronic)2308-457X

    Conference

    ConferenceInterspeech
    Country/TerritoryAustria
    CityGraz
    Period15/09/201919/09/2019
    Internet address

    Keywords

    • Computational modeling
    • L1 acquisition
    • Language acquisition
    • Lexical learning
    • Phonetic learning

    Fingerprint

    Dive into the research topics of 'A computational model of early language acquisition from audiovisual experiences of young infants'. Together they form a unique fingerprint.

    Cite this