Towards Improved Learning in Gaussian Processes: The Best of Two Worlds

Rui Li*, Ti John, Arno Solin

*Corresponding author for this work

Research output: Contribution to conferencePaperScientificpeer-review


Gaussian process training decomposes into inference of the (approximate) posterior and learning of the hyperparameters. For non-Gaussian (non-conjugate) likelihoods, two common choices for approximate inference are Expectation Propagation (EP) and Variational Inference (VI), which have complementary strengths and weaknesses. While VI's lower bound to the marginal likelihood is a suitable objective for inferring the approximate posterior, it does not automatically imply it is a good learning objective for hyperparameter optimization. We design a hybrid training procedure where the inference leverages conjugate-computation VI and the learning uses an EP-like marginal likelihood approximation. We empirically demonstrate on binary classification that this provides a good learning objective and generalizes better.
Original languageEnglish
Publication statusPublished - 2022
MoE publication typeNot Eligible
EventConference on Neural Information Processing Systems - New Orleans, United States
Duration: 28 Nov 20229 Dec 2022
Conference number: 36


ConferenceConference on Neural Information Processing Systems
Abbreviated titleNeurIPS
Country/TerritoryUnited States
CityNew Orleans
Internet address


Dive into the research topics of 'Towards Improved Learning in Gaussian Processes: The Best of Two Worlds'. Together they form a unique fingerprint.

Cite this