Latent Derivative Bayesian Last Layer Networks

Joe Watson*, Jihao Andreas Lin, Pascal Klink, Joni Pajarinen, Jan Peters

*Tämän työn vastaava kirjoittaja

Tutkimustuotos: Artikkeli kirjassa/konferenssijulkaisussaConference contributionScientificvertaisarvioitu

6 Lataukset (Pure)

Abstrakti

Bayesian neural networks (BNN) are powerful parametric models for nonlinear regression with uncertainty quantification. However, the approximate inference techniques for weight space priors suffer from several drawbacks. The 'Bayesian last layer' (BLL) is an alternative BNN approach that learns the feature space for an exact Bayesian linear model with explicit predictive distributions. However, its predictions outside of the data distribution (OOD) are typically overconfident, as the marginal likelihood objective results in a learned feature space that overfits to the data. We overcome this weakness by introducing a functional prior on the model's derivatives w.r.t. the inputs. Treating these Jacobians as latent variables, we incorporate the prior into the objective to influence the smoothness and diversity of the features, which enables greater predictive uncertainty. For the BLL, the Jacobians can be computed directly using forward mode automatic differentiation, and the distribution over Jacobians may be obtained in closed-form. We demonstrate this method enhances the BLL to Gaussian process-like performance on tasks where calibrated uncertainty is critical: OOD regression, Bayesian optimization and active learning, which include high-dimensional real-world datasets.

AlkuperäiskieliEnglanti
Otsikko24TH INTERNATIONAL CONFERENCE ON ARTIFICIAL INTELLIGENCE AND STATISTICS (AISTATS)
ToimittajatA Banerjee, K Fukumizu
KustantajaJMLR
Sivut1198-1206
Sivumäärä13
TilaJulkaistu - 2021
OKM-julkaisutyyppiA4 Artikkeli konferenssijulkaisuussa
TapahtumaInternational Conference on Artificial Intelligence and Statistics - Virtual, Online
Kesto: 13 huhtikuuta 202115 huhtikuuta 2021
Konferenssinumero: 24

Julkaisusarja

NimiProceedings of Machine Learning Research
Vuosikerta130
ISSN (painettu)2640-3498

Conference

ConferenceInternational Conference on Artificial Intelligence and Statistics
LyhennettäAISTATS
KaupunkiVirtual, Online
Ajanjakso13/04/202115/04/2021

Sormenjälki

Sukella tutkimusaiheisiin 'Latent Derivative Bayesian Last Layer Networks'. Ne muodostavat yhdessä ainutlaatuisen sormenjäljen.

Siteeraa tätä