Gaussian process training decomposes into inference of the (approximate) posterior and learning of the hyperparameters. For non-Gaussian (non-conjugate) likelihoods, two common choices for approximate inference are Expectation Propagation (EP) and Variational Inference (VI), which have complementary strengths and weaknesses. While VI's lower bound to the marginal likelihood is a suitable objective for inferring the approximate posterior, it does not automatically imply it is a good learning objective for hyperparameter optimization. We design a hybrid training procedure where the inference leverages conjugate-computation VI and the learning uses an EP-like marginal likelihood approximation. We empirically demonstrate on binary classification that this provides a good learning objective and generalizes better.
|Publication status||Published - 2022|
|MoE publication type||Not Eligible|
|Event||Conference on Neural Information Processing Systems - New Orleans, United States|
Duration: 28 Nov 2022 → 9 Dec 2022
Conference number: 36
|Conference||Conference on Neural Information Processing Systems|
|Period||28/11/2022 → 09/12/2022|