Leave-one-out cross-validation for Bayesian model comparison in large data

Måns Magnusson*, Michael Riis Andersen, Johan Jonasson, Aki Vehtari

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionScientificpeer-review

Abstract

Recently, new methods for model assessment, based on subsampling and posterior approximations, have been proposed for scaling leave-one-out cross-validation (LOO-CV) to large datasets. Although these methods work well for estimating predictive performance for individual models, they are less powerful in model comparison. We propose an efficient method for estimating differences in predictive performance by combining fast approximate LOO surrogates with exact LOO sub-sampling using the difference estimator and supply proofs with regards to scaling characteristics. The resulting approach can be orders of magnitude more efficient than previous approaches, as well as being better suited to model comparison.
Original languageEnglish
Title of host publicationProceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, PMLR
Pages341-351
Publication statusPublished - 2020
MoE publication typeA4 Article in a conference publication
EventInternational Conference on Artificial Intelligence and Statistics - Palermo, Italy
Duration: 3 Jun 20205 Jun 2020
Conference number: 23

Publication series

NameProceedings of Machine Learning Research
PublisherPMLR
Volume108
ISSN (Electronic)2640-3498

Conference

ConferenceInternational Conference on Artificial Intelligence and Statistics
Abbreviated titleAISTATS
Country/TerritoryItaly
CityPalermo
Period03/06/202005/06/2020

Fingerprint

Dive into the research topics of 'Leave-one-out cross-validation for Bayesian model comparison in large data'. Together they form a unique fingerprint.

Cite this