Bayesian Hierarchical Stacking: Some Models Are (Somewhere) Useful

Yuling Yao; Gregor Pirš; Aki Vehtari; Andrew Gelman

doi:10.1214/21-BA1287

Bayesian Hierarchical Stacking: Some Models Are (Somewhere) Useful

Yuling Yao^*
, Gregor Pirš
, Aki Vehtari
, Andrew Gelman

^*Tämän työn vastaava kirjoittaja

Simons Foundation
University of Ljubljana
Columbia University

Tutkimustuotos: Lehtiartikkeli › Article › Scientific › vertaisarvioitu

32 Sitaatiot (Scopus)

316 Lataukset (Pure)

Abstrakti

Stacking is a widely used model averaging technique that asymptotically yields optimal predictions among linear averages. We show that stacking is most effective when model predictive performance is heterogeneous in inputs, and we can further improve the stacked mixture with a hierarchical model. We generalize stacking to Bayesian hierarchical stacking. The model weights are varying as a function of data, partially-pooled, and inferred using Bayesian inference. We further incorporate discrete and continuous inputs, other structured priors, and time series and longitudinal data. To verify the performance gain of the proposed method, we derive theory bounds, and demonstrate on several applied problems.

Alkuperäiskieli	Englanti
Sivut	1043-1071
Sivumäärä	29
Julkaisu	Bayesian Analysis
Vuosikerta	17
Numero	4
DOI - pysyväislinkit	https://doi.org/10.1214/21-BA1287
Tila	Julkaistu - jouluk. 2022
OKM-julkaisutyyppi	A1 Alkuperäisartikkeli tieteellisessä aikakauslehdessä

Rahoitus

∗The authors thank the National Science Foundation, Institute of Education Sciences, Office of Naval Research, National Institutes of Health, Sloan Foundation, Schmidt Futures, and the Academy of Finland Flagship programme: Finnish Center for Artificial Intelligence (FCAI) for partial financial support. Gregor Pirˇs is supported by the Slovenian Research Agency young researcher grant. †Flatiron Institute, New York, USA, [email protected] ‡Faculty of Computer and Information Science, University of Ljubljana, Ljubljana, Slovenia §Department of Computer Science, Aalto University, Espoo, Finland ¶Department of Statistics and Political Science, Columbia University, New York, USA

Pääsy asiakirjaan

10.1214/21-BA1287Lisenssi: CC BY

Bayesian Hierarchical StackingFinal published version, 1,08 MBLisenssi: CC BY

Muut tiedostot ja linkit

Linkki julkaisuun Scopuksessa

Siteeraa tätä

@article{26431c04d548402e98c8c8ea3c265a96,

title = "Bayesian Hierarchical Stacking: Some Models Are (Somewhere) Useful",

abstract = "Stacking is a widely used model averaging technique that asymptotically yields optimal predictions among linear averages. We show that stacking is most effective when model predictive performance is heterogeneous in inputs, and we can further improve the stacked mixture with a hierarchical model. We generalize stacking to Bayesian hierarchical stacking. The model weights are varying as a function of data, partially-pooled, and inferred using Bayesian inference. We further incorporate discrete and continuous inputs, other structured priors, and time series and longitudinal data. To verify the performance gain of the proposed method, we derive theory bounds, and demonstrate on several applied problems.",

keywords = "Bayesian hierarchical modeling, Conditional prediction, Covariate shift, Model averaging, Prior construction, Stacking",

author = "Yuling Yao and Gregor Pir{\v s} and Aki Vehtari and Andrew Gelman",

note = "Publisher Copyright: {\textcopyright} 2022 International Society for Bayesian Analysis",

year = "2022",

month = dec,

doi = "10.1214/21-BA1287",

language = "English",

volume = "17",

pages = "1043--1071",

journal = "Bayesian Analysis",

issn = "1936-0975",

publisher = "International Society for Bayesian Analysis",

number = "4",

}

TY - JOUR

T1 - Bayesian Hierarchical Stacking: Some Models Are (Somewhere) Useful

AU - Yao, Yuling

AU - Pirš, Gregor

AU - Vehtari, Aki

AU - Gelman, Andrew

PY - 2022/12

Y1 - 2022/12

N2 - Stacking is a widely used model averaging technique that asymptotically yields optimal predictions among linear averages. We show that stacking is most effective when model predictive performance is heterogeneous in inputs, and we can further improve the stacked mixture with a hierarchical model. We generalize stacking to Bayesian hierarchical stacking. The model weights are varying as a function of data, partially-pooled, and inferred using Bayesian inference. We further incorporate discrete and continuous inputs, other structured priors, and time series and longitudinal data. To verify the performance gain of the proposed method, we derive theory bounds, and demonstrate on several applied problems.

AB - Stacking is a widely used model averaging technique that asymptotically yields optimal predictions among linear averages. We show that stacking is most effective when model predictive performance is heterogeneous in inputs, and we can further improve the stacked mixture with a hierarchical model. We generalize stacking to Bayesian hierarchical stacking. The model weights are varying as a function of data, partially-pooled, and inferred using Bayesian inference. We further incorporate discrete and continuous inputs, other structured priors, and time series and longitudinal data. To verify the performance gain of the proposed method, we derive theory bounds, and demonstrate on several applied problems.

KW - Bayesian hierarchical modeling

KW - Conditional prediction

KW - Covariate shift

KW - Model averaging

KW - Prior construction

KW - Stacking

UR - https://www.scopus.com/pages/publications/85139205468

U2 - 10.1214/21-BA1287

DO - 10.1214/21-BA1287

M3 - Article

AN - SCOPUS:85139205468

SN - 1936-0975

VL - 17

SP - 1043

EP - 1071

JO - Bayesian Analysis

JF - Bayesian Analysis

IS - 4

ER -

Bayesian Hierarchical Stacking: Some Models Are (Somewhere) Useful

Abstrakti

Rahoitus

Pääsy asiakirjaan

Muut tiedostot ja linkit

Sormenjälki

Siteeraa tätä