Advances in Approximate Bayesian Inference for Gaussian Process Models

Jaakko Riihimäki

    Research output: ThesisDoctoral ThesisCollection of Articles


    Gaussian processes (GPs) provide a flexible approach to construct probabilistic models for Bayesian data analysis. In the Bayesian approach, GPs are used to specify prior assumptions on the latent function values that describe the underlying relationships between the explanatory variables and the associated target variables. These prior assumptions are combined with information from the observations using Bayes' rule. The obtained result is the posterior distribution that represents the uncertainty about the latent function values of interest, conditioned on the observations and the model assumptions. A challenge with the Bayesian approach is that exact inference is analytically intractable to calculate for most GP models used in practice. Therefore, approximate methods are needed in order to evaluate the posterior distribution and to make predictions for new observations. This thesis develops methods for approximate Bayesian inference in various modelling problems involving GP models. The focus is on efficient ways to form Gaussian posterior approximations based on Laplace's method or expectation propagation (EP). The inference for the studied GP models is challenging in two aspects. Firstly, observation models are generalized in the way that the probability distribution for each observation can depend on multiple values of the latent function instead of only one value, or on the derivative values of the latent function. Secondly, instead of one prior process, the models can have multiple uncorrelated prior processes that are coupled through the observation model. This thesis presents improvements to approximate Bayesian inference for GP models in density estimation, survival analysis, and multiclass classification. We describe Laplace's method for a logistic GP model and for a Cox-type survival model constructed from GP priors to speed up the inference. We develop a novel nested EP algorithm for multinomial probit GP classification that does not require numerical quadratures and scales linearly in the number of classes. In addition, we extend the existing methodology proposed for regression and binary classification by introducing monotonicity information into a GP model with EP. We demonstrate the practical accuracy of the described methods with several experiments and apply them to real-life modelling problems.
    Translated title of the contributionMenetelmiä likimääräiseen bayesilaiseen päättelyyn gaussisia prosesseja käyttäville malleille
    Original languageEnglish
    QualificationDoctor's degree
    Awarding Institution
    • Aalto University
    • Lampinen, Jouko, Supervisor
    • Vehtari, Aki, Advisor
    Print ISBNs978-952-60-5332-5
    Electronic ISBNs978-952-60-5333-2
    Publication statusPublished - 2013
    MoE publication typeG5 Doctoral dissertation (article)


    • Bayesian modelling
    • approximate inference
    • Gaussian processes
    • Laplace's method
    • expectation propagation

    Fingerprint Dive into the research topics of 'Advances in Approximate Bayesian Inference for Gaussian Process Models'. Together they form a unique fingerprint.

    Cite this