Variational Fourier Features for Gaussian Processes

James Hensman*, Nicolas Durrande, Arno Solin

*Corresponding author for this work

Research output: Contribution to journalArticleScientificpeer-review

29 Downloads (Pure)

Abstract

This work brings together two powerful concepts in Gaussian processes: the variational approach to sparse approximation and the spectral representation of Gaussian processes. This gives rise to an approximation that inherits the benefits of the variational approach but with the representational power and computational scalability of spectral representations. The work hinges on a key result that there exist spectral features related to a finite domain of the Gaussian process which exhibit almost-independent covariances. We derive these expressions for Matern kernels in one dimension, and generalize to more dimensions using kernels with specific structures. Under the assumption of additive Gaussian noise, our method requires only a single pass through the data set, making for very fast and accurate computation. We fit a model to 4 million training points in just a few minutes on a standard laptop. With non-conjugate likelihoods, our MCMC scheme reduces the cost of computation from O(NM2) (for a sparse Gaussian process) to O(NM) per iteration, where N is the number of data and M is the number of features.

Original languageEnglish
Article number151
Pages (from-to)1-52
Number of pages52
JournalJournal of Machine Learning Research
Volume18
Issue number1
Publication statusPublished - 2018
MoE publication typeA1 Journal article-refereed

Keywords

  • Gaussian processes
  • Fourier features
  • variational inference
  • PROCESS REGRESSION
  • COX PROCESSES
  • APPROXIMATION
  • MODELS

Projects

Sequential inference for real-time probabilistic modelling

Solin, A.

01/09/201724/09/2020

Project: Academy of Finland: Other research funding

Equipment

Science-IT

Mikko Hakala (Manager)

School of Science

Facility/equipment: Facility

  • Cite this