Stochastic Differential Equation Methods for Spatio-Temporal Gaussian Process Regression

Arno Solin

Stochastic Differential Equation Methods for Spatio-Temporal Gaussian Process Regression

Julkaisun otsikon käännös: Stokastisia differentiaaliyhtälömenetelmiä spatiotemporaaliseen regressioon gaussisilla prosesseilla

Arno Solin

Tutkimustuotos: Doctoral Thesis › Collection of Articles

Abstrakti

Gaussiset prosessit (GP) ovat ei-parametrisia tilastollisia työkaluja, joita käytetään yleisesti koneoppimisessa, spatiaalisessa tilastotieteessä sekä signaalinkäsittelyssä. Niiden avulla mallifunktioita koskevat oletukset voidaan määritellä joustavasti. Gaussisen prosessin voi ajatella määrittelevän todennäköisyysjakauman funktioavaruuden yli. Signaalinkäsittelyssä gaussiset prosessit määritellään yleensä tila-avaruusmallin avulla, kun taas koneoppimisessa suositaan ytimeen (kovarianssifunktio) perustuvaa esitystapaa. Jälkimmäisessä esitystavassa GP-regressio-ongelman ratkaisun laskennallinen vaativuus skaalautuu kuutiollisesti aineiston koon suhteen, mikä tekee lähestymistavasta laskennallisesti raskaan suurilla tietoaineistoilla. Tässä työssä tarkastellaan tila-avaruusmallien ja kovarianssifunktioiden yhteyttä, jonka avulla voidaan käyttää tilastollisessa päättelyssä tehokkaita vaiheittaisia Kalman-suotimeen perustuvia menetelmiä. Nämä menetelmät mahdollistavat lineaarisen laskennallisen skaalautuvuuden aineiston koon suhteen. Aluksi yhteys esitetään yksiulotteisille (temporaalisille) kovarianssifunktioille, jotka voidaan esittää lineaarisina aikainvariantteina stokastisina differentiaaliyhtälöinä. Käytetyt kovarianssifunktiot sisältävät sekä stationaarisia että ei-stationaarisia GP-malleja, joiden avulla mallioletuksiin voidaan sisällyttää esimerkiksi jatkuvuutta, sileyttä tai jaksollisuutta. Tämän jälkeen menetelmäkehystä laajennetaan spatiotemporaalisiin malleihin, joissa GP esitetään evoluutiotyyppisenä stokastisena osittaisdifferentiaaliyhtälönä ja päättely tehdään ääretönulotteista Kalman-suodinta käyttämällä. Työssä käsitellään separoituvia ja ei-separoituvia malleja sekä tarkastellaan laskennallisia toteutustapoja. Stokastisten differentiaaliyhtälöiden ja yleisesti käytettyjen kovarianssifunktioiden yhteyttä voidaan käyttää myös GP-mallien yhdistämisessä mekanistisiin fysikaalisiin differentiaaliyhtälöihin. Temporaalisia ja spatiotemporaalisia gaussisia prosesseja voidaan käyttää monenlaisissa tietomääriltään suurissa sovelluksissa. Tässä työssä käytetään esimerkkeinä sovelluksia aivokuvannuksen, sään mallintamisen, markkinoiden ennustamisen sekä paikannuksen aloilta.

Julkaisun otsikon käännös	Stokastisia differentiaaliyhtälömenetelmiä spatiotemporaaliseen regressioon gaussisilla prosesseilla
Alkuperäiskieli	Englanti
Pätevyys	Tohtorintutkinto
Myöntävä instituutio	Aalto-yliopisto
Valvoja/neuvonantaja	Lampinen, Jouko, Vastuuprofessori Särkkä, Simo, Ohjaaja
Kustantaja	Aalto University
Painoksen ISBN	978-952-60-6710-0
Sähköinen ISBN	978-952-60-6711-7
Tila	Julkaistu - 2016
OKM-julkaisutyyppi	G5 Artikkeliväitöskirja

Tutkimusalat

stokastinen differentiaaliyhtälö
gaussinen prosessi
tila-avaruusmalli
spatiotemporaaliaineisto

Pääsy asiakirjaan

http://urn.fi/URN:ISBN:978-952-60-6711-7

Siteeraa tätä

@phdthesis{d0b53841a4f64983bf3b9ace803e5d1f,

title = "Stochastic Differential Equation Methods for Spatio-Temporal Gaussian Process Regression",

abstract = "Gaussian processes (GPs) are widely used tools for non-parametric probabilistic modelling in machine learning, spatial statistics, and signal processing. Their strength lies in flexible model specification, where prior beliefs of the model functions are encoded by the GP model. This way they can also be interpreted as specifying a probability distribution over the space of functions. In signal processing GPs are typically represented as state-space models, whereas the kernel (covariance function) representation is favoured in machine learning. Under the kernel formalism, the na{\"i}ve solution to a GP regression problem scales cubically in the number of data points, which makes the approach computationally infeasible for large data sets. This work explores the link between the two representations, which enables the use of efficient sequential Kalman filtering based methods for solving the inference problem. These methods have linear time complexity with respect to the number of data points. The interest is in presenting an explicit connection between a large class of covariance functions and state- space models. This is done for one-dimensional (temporal) covariance functions and linear time-invariant stochastic differential equations. This class of models covers a wide range of both stationary and non-stationary GP models for encoding, for example, continuity, smoothness, or periodicity. The framework also extends to spatio-temporal models, where the GP is represented as an evolution type stochastic partial differential equation and inference conducted by infinite-dimensional Kalman filtering methods. Both separable and non- separable models are considered, and implementation techniques for numerical solutions are discussed. The link between stochastic differential equations and standard covariance functions widens the applicability of Gaussian processes in combination with mechanistic physical differential equation models. Temporal and spatio-temporal Gaussian process models are useful in a multitude of data-intensive applications. Examples in this work include brain image analysis, weather modelling, financial forecasting, and tracking applications.",

keywords = "stochastic differential equation, Gaussian process, state-space model, spatio-temporal data, stokastinen differentiaaliyht{\"a}l{\"o}, gaussinen prosessi, tila-avaruusmalli, spatiotemporaaliaineisto, stochastic differential equation, Gaussian process, state-space model, spatio-temporal data",

author = "Arno Solin",

year = "2016",

language = "English",

isbn = "978-952-60-6710-0",

series = "Aalto University publication series DOCTORAL DISSERTATIONS",

publisher = "Aalto University",

number = "50",

address = "Finland",

school = "Aalto University",

}

TY - BOOK

T1 - Stochastic Differential Equation Methods for Spatio-Temporal Gaussian Process Regression

AU - Solin, Arno

PY - 2016

Y1 - 2016

N2 - Gaussian processes (GPs) are widely used tools for non-parametric probabilistic modelling in machine learning, spatial statistics, and signal processing. Their strength lies in flexible model specification, where prior beliefs of the model functions are encoded by the GP model. This way they can also be interpreted as specifying a probability distribution over the space of functions. In signal processing GPs are typically represented as state-space models, whereas the kernel (covariance function) representation is favoured in machine learning. Under the kernel formalism, the naïve solution to a GP regression problem scales cubically in the number of data points, which makes the approach computationally infeasible for large data sets. This work explores the link between the two representations, which enables the use of efficient sequential Kalman filtering based methods for solving the inference problem. These methods have linear time complexity with respect to the number of data points. The interest is in presenting an explicit connection between a large class of covariance functions and state- space models. This is done for one-dimensional (temporal) covariance functions and linear time-invariant stochastic differential equations. This class of models covers a wide range of both stationary and non-stationary GP models for encoding, for example, continuity, smoothness, or periodicity. The framework also extends to spatio-temporal models, where the GP is represented as an evolution type stochastic partial differential equation and inference conducted by infinite-dimensional Kalman filtering methods. Both separable and non- separable models are considered, and implementation techniques for numerical solutions are discussed. The link between stochastic differential equations and standard covariance functions widens the applicability of Gaussian processes in combination with mechanistic physical differential equation models. Temporal and spatio-temporal Gaussian process models are useful in a multitude of data-intensive applications. Examples in this work include brain image analysis, weather modelling, financial forecasting, and tracking applications.

AB - Gaussian processes (GPs) are widely used tools for non-parametric probabilistic modelling in machine learning, spatial statistics, and signal processing. Their strength lies in flexible model specification, where prior beliefs of the model functions are encoded by the GP model. This way they can also be interpreted as specifying a probability distribution over the space of functions. In signal processing GPs are typically represented as state-space models, whereas the kernel (covariance function) representation is favoured in machine learning. Under the kernel formalism, the naïve solution to a GP regression problem scales cubically in the number of data points, which makes the approach computationally infeasible for large data sets. This work explores the link between the two representations, which enables the use of efficient sequential Kalman filtering based methods for solving the inference problem. These methods have linear time complexity with respect to the number of data points. The interest is in presenting an explicit connection between a large class of covariance functions and state- space models. This is done for one-dimensional (temporal) covariance functions and linear time-invariant stochastic differential equations. This class of models covers a wide range of both stationary and non-stationary GP models for encoding, for example, continuity, smoothness, or periodicity. The framework also extends to spatio-temporal models, where the GP is represented as an evolution type stochastic partial differential equation and inference conducted by infinite-dimensional Kalman filtering methods. Both separable and non- separable models are considered, and implementation techniques for numerical solutions are discussed. The link between stochastic differential equations and standard covariance functions widens the applicability of Gaussian processes in combination with mechanistic physical differential equation models. Temporal and spatio-temporal Gaussian process models are useful in a multitude of data-intensive applications. Examples in this work include brain image analysis, weather modelling, financial forecasting, and tracking applications.

KW - stochastic differential equation

KW - Gaussian process

KW - state-space model

KW - spatio-temporal data

KW - stokastinen differentiaaliyhtälö

KW - gaussinen prosessi

KW - tila-avaruusmalli

KW - spatiotemporaaliaineisto

KW - stochastic differential equation

KW - Gaussian process

KW - state-space model

KW - spatio-temporal data

M3 - Doctoral Thesis

SN - 978-952-60-6710-0

T3 - Aalto University publication series DOCTORAL DISSERTATIONS

PB - Aalto University

ER -

Stochastic Differential Equation Methods for Spatio-Temporal Gaussian Process Regression

Abstrakti

Tutkimusalat

Pääsy asiakirjaan

Sormenjälki

Siteeraa tätä