Leave-one-out cross-validation for Bayesian model comparison in large data

Måns Magnusson*, Michael Riis Andersen, Johan Jonasson, Aki Vehtari

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionScientificpeer-review


Recently, new methods for model assessment, based on subsampling and posterior approximations, have been proposed for scaling leave-one-out cross-validation (LOO-CV) to large datasets. Although these methods work well for estimating predictive performance for individual models, they are less powerful in model comparison. We propose an efficient method for estimating differences in predictive performance by combining fast approximate LOO surrogates with exact LOO sub-sampling using the difference estimator and supply proofs with regards to scaling characteristics. The resulting approach can be orders of magnitude more efficient than previous approaches, as well as being better suited to model comparison.
Original languageEnglish
Title of host publicationProceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, PMLR
Publication statusPublished - 2020
MoE publication typeA4 Article in a conference publication
EventInternational Conference on Artificial Intelligence and Statistics - Palermo, Italy
Duration: 3 Jun 20205 Jun 2020
Conference number: 23

Publication series

NameProceedings of Machine Learning Research
ISSN (Electronic)2640-3498


ConferenceInternational Conference on Artificial Intelligence and Statistics
Abbreviated titleAISTATS


Dive into the research topics of 'Leave-one-out cross-validation for Bayesian model comparison in large data'. Together they form a unique fingerprint.

Cite this