Methods for Interpreting Kernel Canonical Correlation Measures

Viivi Uurtio

Methods for Interpreting Kernel Canonical Correlation Measures

Julkaisun otsikon käännös: Menetelmiä ydinfunktioilla laajennettujen riippuvuusmittojen tulkintaan

Viivi Uurtio

Tutkimustuotos: Doctoral Thesis › Collection of Articles

Abstrakti

Luonnossa esiintyy ei-monotonisia riippuvuuksia sisältäviä monimuuttujasysteemeitä. Neuro- ja taloustieteissä ilmenee esimerkkejä tällaisista systeemeistä, joissa useampi prosessi eli muuttujien yhdistelmä tapahtuu samanaikaisesti, mutta ei välttämättä lineaarisesti tai monotonisesti. Tämänkaltaissa systeemeissä esiintyviä ei-monotonisia riippuvuuksia voidaan mitata ydinfunktioilla laajennetulla kanonisella korrelaatiolla ja Hilbert-Schmidt riippumattomuuskriteerillä. Nämä mitat kertovat kuitenkin vain, jos riippuvuus esiintyy systeemissä, mutta niiden arvot eivät sellaisenaan riitä kuvaamaan, mitkä prosessit eli muuttujat ovat riippuvaisia toisistaan. Tässä väitöskirjassa tarkastellaan ydinfunktiolla laajennetun kanonisen korrelaation ja Hilbert-Schmidt riippumattomuuskriteerin tulkintaa. Aluksi tutkimme, miten ydinfunktiolla laajennettua kanonista korrelaatiota on aiemmin kirjallisuudessa tulkittu. Tämän jälkeen kehitämme kirjallisuuden menetelmää hierarkisella klusteroinnilla. Sitten esitämme uuden vuorottaisiin projisoituihin gradientteihin perustuvan menetelmän eli gradKCCA:n, joka laskee sekä ydinfunktiolla laajennetun kanonisen korrelaation arvon ja muutujien väliset riippuvuudet. Lopuksi esitämme uuden vuorottaisiin stokastisiin projisoituihin gradientteihin perustuvan algoritmin nimeltä SCCA-HSIC, johon voidaan myös yhdistää Nyström approksimaatio. SCCA-HSIC optimoi Hilbert-Schmidt riippumattomuuskriteeriä suhteessa muuttujien välisiin riippuvuuksiin. Ensimmäisessä julkaisussa poimimme kirjallisuudesta korrelaatioihin perustuvan menetelmän ydinfunktiolla laajennetun kanonisen korrelaation tulkitsemiseksi. Toisessa julkaisussa esitämme klustergram-visualisoinnin, joka on yhtenäinen kuva kirjallisuuden korrelaatio-menetelmän arvoista. Klustergrammia sovellettiin syvän peruskallion pohjavesistöstä kerätyn datan analyysissä. Kolmannessa julkaisussa näytetään, että gradKCCA on nopeampi, skaalautuvampi ja tarkempi menetelmä riippuvuuksien määrittelyssä verrattuna viimeisimpiin menetelmiin. Kokeissa käytetään simuloituja ja aitoja datajoukkoja. Neljännessä julkaisussa näytetään, että SCCA-HSIC on tarkempi ja skaalautuvampi menetelmä kuin viimeisimmät menetelmät, kun arvioimme sitä simuloiduilla ja aidoilla datajoukoilla. Esitetyt menetelmät ja algoritmit parantavat mahdollisten ei-monotonisten riippuvuuksien tulkintaa. Ne ovat helppokäyttöisiä ja sopivat niille, jotka haluaisivat ymmärtää jonkin monimuuttujasysteemin toiminnan. Menetelmiä voi myös soveltaa laajemmin muihinkin ydinfunktioihin perustuviin riippuvuusmittoihin.

Julkaisun otsikon käännös	Menetelmiä ydinfunktioilla laajennettujen riippuvuusmittojen tulkintaan
Alkuperäiskieli	Englanti
Pätevyys	Tohtorintutkinto
Myöntävä instituutio	Aalto-yliopisto
Valvoja/neuvonantaja	Rousu, Juho, Vastuuprofessori Bhadra, Sahely, Ohjaaja
Kustantaja	Aalto University
Painoksen ISBN	978-952-60-8963-8
Sähköinen ISBN	978-952-60-8964-5
Tila	Julkaistu - 2020
OKM-julkaisutyyppi	G5 Artikkeliväitöskirja

Tutkimusalat

kanoninen korrelaatio
ydinfunktiomenetelmät
skaalautuvuus
harvuus

Pääsy asiakirjaan

http://urn.fi/URN:ISBN:978-952-60-8964-5

Siteeraa tätä

@phdthesis{3af473e6ce52463c8903f5613c18a385,

title = "Methods for Interpreting Kernel Canonical Correlation Measures",

abstract = "In nature, we can find multivariate systems that contain non-monotone dependencies. Examples of these systems arise in neuroscience and economics, where several processes, that is combinations of variables, co-occur together but not necessarily linearly or monotonically. Kernel canonical correlation measures, that is the kernel canonical correlation and the Hilbert-Schmidt independence criterion, can be applied to capture the non-monotone dependencies occurring in these types of systems. However, in general, these measures convey whether a dependence exists in the system but it may not be straightforward to determine which of the processes, or variables, in the system are dependent. This thesis addresses the problem of interpreting kernel canonical correlation measures. First, we review the literature to identify the proposed strategies for interpreting the kernel canonical correlation. Second, we further extend the already presented technique by applying hierarchical clustering. Third, we propose a novel alternating projected gradient approach, gradKCCA, to compute both the kernel canonical correlation and the dependencies. Fourth, we present a novel alternating stochastic projected gradient algorithm, SCCA-HSIC, that can be combined with the Nystr{\"o}m approximation, to optimise the Hilbert-Schmidt independence criterion with respect to the dependencies. The first article identifies a correlation-based strategy for interpreting the kernel canonical correlation. The second article presents a clustergram visualisation, that summarises the correlations obtained by the first technique, on a heatmap. The clustergram visualisation is demonstrated on a real-world dataset obtained from deep bedrock groundwaters. In the third article, gradKCCA is shown to be fast, scalable, and accurate in identifying the dependencies in relation to state-of-the-art methods, both in simulation studies and in real-world datasets. The contribution of the fourth article, SCCA-HSIC, is shown to have a superior accuracy and scalability than the state-of-the-art methods, when evaluated on simulated and real-world datasets. The proposed methods and algorithms provide tools for better interpretation of the underlying, possibly non-monotone, dependent processes. These tools can easily be deployed by practitioners seeking to understand the functioning of a multivariate system. Finally, the presented optimisation strategies can be considered for alternative kernel-based dependence measures.",

keywords = "canonical correlation, kernel methods, scalability, sparsity, kanoninen korrelaatio, ydinfunktiomenetelm{\"a}t, skaalautuvuus, harvuus, canonical correlation, kernel methods, scalability, sparsity",

author = "Viivi Uurtio",

year = "2020",

language = "English",

isbn = "978-952-60-8963-8",

series = "Aalto University publication series DOCTORAL DISSERTATIONS",

publisher = "Aalto University",

number = "21",

address = "Finland",

school = "Aalto University",

}

TY - BOOK

T1 - Methods for Interpreting Kernel Canonical Correlation Measures

AU - Uurtio, Viivi

PY - 2020

Y1 - 2020

N2 - In nature, we can find multivariate systems that contain non-monotone dependencies. Examples of these systems arise in neuroscience and economics, where several processes, that is combinations of variables, co-occur together but not necessarily linearly or monotonically. Kernel canonical correlation measures, that is the kernel canonical correlation and the Hilbert-Schmidt independence criterion, can be applied to capture the non-monotone dependencies occurring in these types of systems. However, in general, these measures convey whether a dependence exists in the system but it may not be straightforward to determine which of the processes, or variables, in the system are dependent. This thesis addresses the problem of interpreting kernel canonical correlation measures. First, we review the literature to identify the proposed strategies for interpreting the kernel canonical correlation. Second, we further extend the already presented technique by applying hierarchical clustering. Third, we propose a novel alternating projected gradient approach, gradKCCA, to compute both the kernel canonical correlation and the dependencies. Fourth, we present a novel alternating stochastic projected gradient algorithm, SCCA-HSIC, that can be combined with the Nyström approximation, to optimise the Hilbert-Schmidt independence criterion with respect to the dependencies. The first article identifies a correlation-based strategy for interpreting the kernel canonical correlation. The second article presents a clustergram visualisation, that summarises the correlations obtained by the first technique, on a heatmap. The clustergram visualisation is demonstrated on a real-world dataset obtained from deep bedrock groundwaters. In the third article, gradKCCA is shown to be fast, scalable, and accurate in identifying the dependencies in relation to state-of-the-art methods, both in simulation studies and in real-world datasets. The contribution of the fourth article, SCCA-HSIC, is shown to have a superior accuracy and scalability than the state-of-the-art methods, when evaluated on simulated and real-world datasets. The proposed methods and algorithms provide tools for better interpretation of the underlying, possibly non-monotone, dependent processes. These tools can easily be deployed by practitioners seeking to understand the functioning of a multivariate system. Finally, the presented optimisation strategies can be considered for alternative kernel-based dependence measures.

AB - In nature, we can find multivariate systems that contain non-monotone dependencies. Examples of these systems arise in neuroscience and economics, where several processes, that is combinations of variables, co-occur together but not necessarily linearly or monotonically. Kernel canonical correlation measures, that is the kernel canonical correlation and the Hilbert-Schmidt independence criterion, can be applied to capture the non-monotone dependencies occurring in these types of systems. However, in general, these measures convey whether a dependence exists in the system but it may not be straightforward to determine which of the processes, or variables, in the system are dependent. This thesis addresses the problem of interpreting kernel canonical correlation measures. First, we review the literature to identify the proposed strategies for interpreting the kernel canonical correlation. Second, we further extend the already presented technique by applying hierarchical clustering. Third, we propose a novel alternating projected gradient approach, gradKCCA, to compute both the kernel canonical correlation and the dependencies. Fourth, we present a novel alternating stochastic projected gradient algorithm, SCCA-HSIC, that can be combined with the Nyström approximation, to optimise the Hilbert-Schmidt independence criterion with respect to the dependencies. The first article identifies a correlation-based strategy for interpreting the kernel canonical correlation. The second article presents a clustergram visualisation, that summarises the correlations obtained by the first technique, on a heatmap. The clustergram visualisation is demonstrated on a real-world dataset obtained from deep bedrock groundwaters. In the third article, gradKCCA is shown to be fast, scalable, and accurate in identifying the dependencies in relation to state-of-the-art methods, both in simulation studies and in real-world datasets. The contribution of the fourth article, SCCA-HSIC, is shown to have a superior accuracy and scalability than the state-of-the-art methods, when evaluated on simulated and real-world datasets. The proposed methods and algorithms provide tools for better interpretation of the underlying, possibly non-monotone, dependent processes. These tools can easily be deployed by practitioners seeking to understand the functioning of a multivariate system. Finally, the presented optimisation strategies can be considered for alternative kernel-based dependence measures.

KW - canonical correlation

KW - kernel methods

KW - scalability

KW - sparsity

KW - kanoninen korrelaatio

KW - ydinfunktiomenetelmät

KW - skaalautuvuus

KW - harvuus

KW - canonical correlation

KW - kernel methods

KW - scalability

KW - sparsity

M3 - Doctoral Thesis

SN - 978-952-60-8963-8

T3 - Aalto University publication series DOCTORAL DISSERTATIONS

PB - Aalto University

ER -

Methods for Interpreting Kernel Canonical Correlation Measures

Abstrakti

Tutkimusalat

Pääsy asiakirjaan

Sormenjälki

Siteeraa tätä