Hamiltonian Monte Carlo using an adjoint-differentiated Laplace approximation: Bayesian inference for latent Gaussian models and beyond

Charles Margossian*, Aki Vehtari, Daniel Simpson, Raj Agrawal

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionScientificpeer-review

Abstract

Gaussian latent variable models are a key class of Bayesian hierarchical models with applications in many fields. Performing Bayesian inference on such models can be challenging as Markov chain Monte Carlo algorithms struggle with the geometry of the resulting posterior distribution and can be prohibitively slow. An alternative is to use a Laplace approximation to marginalize out the latent Gaussian variables and then integrate out the remaining hyperparameters using dynamic Hamiltonian Monte Carlo, a gradient-based Markov chain Monte Carlo sampler. To implement this scheme efficiently, we derive a novel adjoint method that propagates the minimal information needed to construct the gradient of the approximate marginal likelihood. This strategy yields a scalable differentiation method that is orders of magnitude faster than state of the art differentiation techniques when the hyperparameters are high dimensional. We prototype the method in the probabilistic programming framework Stan and test the utility of the embedded Laplace approximation on several models, including one where the dimension of the hyperparameter is ∼6,000. Depending on the cases, the benefits can include an alleviation of the geometric pathologies that frustrate Hamiltonian Monte Carlo and a dramatic speed-up
Original languageEnglish
Title of host publicationThirty-fourth Conference on Neural Information Processing Systems
Number of pages12
Publication statusPublished - 2020
MoE publication typeA4 Article in a conference publication
EventIEEE Conference on Neural Information Processing Systems; - Virtual, Vancouver, Canada
Duration: 6 Dec 202012 Dec 2020
Conference number: 34

Publication series

NameAdvances in Neural Information Processing Systems
PublisherMorgan Kaufmann Publishers
Volume33
ISSN (Electronic)1049-5258

Conference

ConferenceIEEE Conference on Neural Information Processing Systems;
Abbreviated titleNeurIPS
CountryCanada
CityVancouver
Period06/12/202012/12/2020

Fingerprint Dive into the research topics of 'Hamiltonian Monte Carlo using an adjoint-differentiated Laplace approximation: Bayesian inference for latent Gaussian models and beyond'. Together they form a unique fingerprint.

Cite this