Incorporating functional summary information in Bayesian neural networks using a Dirichlet process likelihood approach

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingsScientificpeer-review

28 Downloads (Pure)

Abstract

Bayesian neural networks (BNNs) can account for both aleatoric and epistemic uncertainty. However, in BNNs the priors are often specified over the weights which rarely reflects true prior knowledge in large and complex neural network architectures. We present a simple approach to incorporate prior knowledge in BNNs based on external summary information about the predicted classification probabilities for a given dataset. The available summary information is incorporated as augmented data and modeled with a Dirichlet process, and we derive the corresponding Summary Evidence Lower BOund. The approach is founded on Bayesian principles, and all hyperparameters have a proper probabilistic interpretation. We show how the method can inform the model about task difficulty and class imbalance. Extensive experiments show that, with negligible computational overhead, our method parallels and in many cases outperforms popular alternatives in accuracy, uncertainty calibration, and robustness against corruptions with both balanced and imbalanced data.
Original languageEnglish
Title of host publicationProceedings of The 26th International Conference on Artificial Intelligence and Statistics (AISTATS) 2023
EditorsFrancisco Ruiz, Jennifer Dy, Jan-Willem van de Meent
PublisherJMLR
Pages6741-6763
Publication statusPublished - 2023
MoE publication typeA4 Conference publication
EventInternational Conference on Artificial Intelligence and Statistics - Valencia, Spain
Duration: 25 Apr 202327 Apr 2023
Conference number: 26
http://aistats.org/aistats2023/

Publication series

NameProceedings of Machine Learning Research
PublisherJMLR
Volume206
ISSN (Print)2640-3498

Conference

ConferenceInternational Conference on Artificial Intelligence and Statistics
Abbreviated titleAISTATS
Country/TerritorySpain
CityValencia
Period25/04/202327/04/2023
Internet address

Fingerprint

Dive into the research topics of 'Incorporating functional summary information in Bayesian neural networks using a Dirichlet process likelihood approach'. Together they form a unique fingerprint.

Cite this