Emergence of representations in natural data

Jaakko Väyrynen

Emergence of representations in natural data

Julkaisun otsikon käännös: Esitysten kehkeytyminen luonnollisesta datasta

Jaakko Väyrynen

Tutkimustuotos: Doctoral Thesis › Collection of Articles

Abstrakti

Tässä väitöskirjassa mallinnetaan aineistolähteisesti luonnollisia kuvia ja tekstiä sekä tarkastellaan niistä löydettyjä esitystapoja. Ympäristöstä tulevat ärsykkeet ohjaavat kognitiivista kehitystä ja aivojen toimintaa. Aineistolähtöiset laskennalliset menetelmät tarjoavat vastaavasti joustavan tavan löytää erilaisia eksploratiivisia näkökulmia. Riippumattomien komponenttien analyysi (ICA) on ansioitunut ohjaamaton tilastollinen menetelmä erityisesti hermostollisten vasteiden käsittelyssä. Se pystyy erottamaan kognitiivisesti olennaiset lähteet näennäisesti sekoittuneista signaaleista tilastollisen riippumattomuusoletuksen avulla. Tämä konsepti liittyy läheisesti harvakoodaukseen, joka on neurobiologisesti tehokas ja edustaa näkemystä aistitiedon käsittelystä aivoissa. Työssä sovelletaan myös toista tilastollista käsitettä, temporaalista koherenssia, videoanalyysissä ja verrataan sitä ICA-menetelmään. Löydetyt esitykset mallintavat samoja ominaisuuksia kuin primäärin näköaivokuoren yksittäissolut. Työssä esitetään yhteismalli, joka yhdistää harvuuden, temporaalisen koherenssin ja topologisen järjestymisen. Samanlaisia menetelmällisiä työkaluja sovelletaan myös luonnollisen kielen mallintamiseen. Työssä käytetään pelkistettyä esikäsittelyä kieliriippumattomien menetelmien aikaansaamiseksi. Sanojen merkityksiä mallinnetaan kontekstuaalisten yhteisesiintymien avulla isosta korpuksesta vektoriavaruusmalleilla. ICA-menetelmällä paljastetaan alla oleva harva rakenne ja tehdään esityksistä helpommin tulkittavia verrattuna klassisiin menetelmiin, jotka perustuvat toisen asteen tilastollisiin ominaisuuksiin. Kokeelliset tulokset vahvistavat käytettyjen ohjaamattomien menetelmien toimivuuden ja osoittavat, että aineiston parametrisoinnilla on merkittävä vaikutus löydettyihin esityksiin. Työssä kehitetyillä analyysimenetelmillä havaitaan löydettyjen piirteiden vastaavan eri tasoilla olevia perinteisiä syntaktisia ja semanttisia kategorioita sekä sanojen välisiä suhteita. Tulokset laajenevat myös monikieliseen aineistoon ja löydettyjen sanaesitysten lisääntynyt harvuus vahvistetaan epälineaarisella kynnystyksellä. Työn tuloksia voidaan käyttää rakentamaan distributionaalisia malleja luonnollisen kielen käsittelyn tehtävissä, joissa tulkittavuus on toivottu ominaisuus.

Julkaisun otsikon käännös	Esitysten kehkeytyminen luonnollisesta datasta
Alkuperäiskieli	Englanti
Pätevyys	Tohtorintutkinto
Myöntävä instituutio	Aalto-yliopisto
Ohjaaja	Kurimo, Mikko, Vastuuprofessori Creutz, Mathias, Ohjaaja
Kustantaja	Aalto University
Painoksen ISBN	978-952-60-7583-9
Sähköinen ISBN	978-952-60-7582-2
Tila	Julkaistu - 2017
OKM-julkaisutyyppi	G5 Artikkeliväitöskirja

Tutkimusalat

leksikaalinen semantiikka
näkö
kieli
merkitys
laskennallinen mallintaminen
vektoriavaruusmalli
ohjaamaton oppiminen
kieliriippumattomuus
koneoppiminen

Pääsy asiakirjaan

http://urn.fi/URN:ISBN:978-952-60-7582-2Lisenssi: Määrittelemätön

Siteeraa tätä

@phdthesis{d2dad77e7ec94c5aa294ccef17c9e83a,

title = "Emergence of representations in natural data",

abstract = "This dissertation models natural image and language data with data-driven methods with focus in the interpretation of the emergent representation. Cognitive development and processing learns to handle input from the surrounding environment. Similarly, data-driven methods offer a flexible way to find exploratory views of the data. Independent Component Analysis (ICA) is a proven unsupervised method especially in the field of neural signal processing. It can extract cognitively relevant source signals from seemingly garbled signal mixtures with the assumption of statistical independence. The concept is closely related to sparse coding, which is neurobiologically efficient and is a view of how sensory information is processed in the brain. In the analysis of small video segments, another statistical concept, temporal coherence, is applied and the results are compared to those of ICA. The representations learned share major characteristics with those measured from the early processing in the visual cortex. A unified model which combines sparseness, temporal coherence and topological organization is introduced. With similar methodological tools, the focus is shifted to natural language data with only minimal preprocessing in order to create language-independent methods. The meaning of words can be modeled with contextual co-occurrence information collected from a large corpus and vector space models. In contrast to classical methods utilizing second-order statistics, the ICA method can reveal the underlying sparse structure and make the representation more interpretable. In addition to validating the applied unsupervised methodology, the experimental results indicate that the parametrization of the data has a very large effect on the representation learned. With the developed analysis tools, the structure learned is matched to syntactic and semantic features at different levels. For translated sentence pairs, the result is a multilingual representation for words. The increased sparsity of the representations learned is validated by further nonlinear thresholding. The findings can be utilized to build distributional models for words which match better with semantic theories of word classes and relationships among word meanings in natural language processing tasks where more interpretability is desired.",

keywords = "lexical semantics, vision, language, meaning, computational modeling, vector space models, unsupervised learning, language independence, machine learning, leksikaalinen semantiikka, n{\"a}k{\"o}, kieli, merkitys, laskennallinen mallintaminen, vektoriavaruusmalli, ohjaamaton oppiminen, kieliriippumattomuus, koneoppiminen, lexical semantics, vision, language, meaning, computational modeling, vector space models, unsupervised learning, language independence, machine learning",

author = "Jaakko V{\"a}yrynen",

year = "2017",

language = "English",

isbn = "978-952-60-7583-9",

series = "Aalto University publication series DOCTORAL DISSERTATIONS",

publisher = "Aalto University",

number = "160",

address = "Finland",

school = "Aalto University",

}

TY - BOOK

T1 - Emergence of representations in natural data

AU - Väyrynen, Jaakko

PY - 2017

Y1 - 2017

N2 - This dissertation models natural image and language data with data-driven methods with focus in the interpretation of the emergent representation. Cognitive development and processing learns to handle input from the surrounding environment. Similarly, data-driven methods offer a flexible way to find exploratory views of the data. Independent Component Analysis (ICA) is a proven unsupervised method especially in the field of neural signal processing. It can extract cognitively relevant source signals from seemingly garbled signal mixtures with the assumption of statistical independence. The concept is closely related to sparse coding, which is neurobiologically efficient and is a view of how sensory information is processed in the brain. In the analysis of small video segments, another statistical concept, temporal coherence, is applied and the results are compared to those of ICA. The representations learned share major characteristics with those measured from the early processing in the visual cortex. A unified model which combines sparseness, temporal coherence and topological organization is introduced. With similar methodological tools, the focus is shifted to natural language data with only minimal preprocessing in order to create language-independent methods. The meaning of words can be modeled with contextual co-occurrence information collected from a large corpus and vector space models. In contrast to classical methods utilizing second-order statistics, the ICA method can reveal the underlying sparse structure and make the representation more interpretable. In addition to validating the applied unsupervised methodology, the experimental results indicate that the parametrization of the data has a very large effect on the representation learned. With the developed analysis tools, the structure learned is matched to syntactic and semantic features at different levels. For translated sentence pairs, the result is a multilingual representation for words. The increased sparsity of the representations learned is validated by further nonlinear thresholding. The findings can be utilized to build distributional models for words which match better with semantic theories of word classes and relationships among word meanings in natural language processing tasks where more interpretability is desired.

AB - This dissertation models natural image and language data with data-driven methods with focus in the interpretation of the emergent representation. Cognitive development and processing learns to handle input from the surrounding environment. Similarly, data-driven methods offer a flexible way to find exploratory views of the data. Independent Component Analysis (ICA) is a proven unsupervised method especially in the field of neural signal processing. It can extract cognitively relevant source signals from seemingly garbled signal mixtures with the assumption of statistical independence. The concept is closely related to sparse coding, which is neurobiologically efficient and is a view of how sensory information is processed in the brain. In the analysis of small video segments, another statistical concept, temporal coherence, is applied and the results are compared to those of ICA. The representations learned share major characteristics with those measured from the early processing in the visual cortex. A unified model which combines sparseness, temporal coherence and topological organization is introduced. With similar methodological tools, the focus is shifted to natural language data with only minimal preprocessing in order to create language-independent methods. The meaning of words can be modeled with contextual co-occurrence information collected from a large corpus and vector space models. In contrast to classical methods utilizing second-order statistics, the ICA method can reveal the underlying sparse structure and make the representation more interpretable. In addition to validating the applied unsupervised methodology, the experimental results indicate that the parametrization of the data has a very large effect on the representation learned. With the developed analysis tools, the structure learned is matched to syntactic and semantic features at different levels. For translated sentence pairs, the result is a multilingual representation for words. The increased sparsity of the representations learned is validated by further nonlinear thresholding. The findings can be utilized to build distributional models for words which match better with semantic theories of word classes and relationships among word meanings in natural language processing tasks where more interpretability is desired.

KW - lexical semantics

KW - vision

KW - language

KW - meaning

KW - computational modeling

KW - vector space models

KW - unsupervised learning

KW - language independence

KW - machine learning

KW - leksikaalinen semantiikka

KW - näkö

KW - kieli

KW - merkitys

KW - laskennallinen mallintaminen

KW - vektoriavaruusmalli

KW - ohjaamaton oppiminen

KW - kieliriippumattomuus

KW - koneoppiminen

KW - lexical semantics

KW - vision

KW - language

KW - meaning

KW - computational modeling

KW - vector space models

KW - unsupervised learning

KW - language independence

KW - machine learning

M3 - Doctoral Thesis

SN - 978-952-60-7583-9

T3 - Aalto University publication series DOCTORAL DISSERTATIONS

PB - Aalto University

ER -

Emergence of representations in natural data

Abstrakti

Tutkimusalat

Pääsy asiakirjaan

Sormenjälki

Siteeraa tätä