Projects per year
Abstract
Approximate Bayesian inference estimates descriptors of an intractable target distribution - in essence, an optimization problem within a family of distributions. For example, Langevin dynamics (LD) extracts asymptotically exact samples from a diffusion process because the time evolution of its marginal distributions constitutes a curve that minimizes the KL-divergence via steepest descent in the Wasserstein space. Parallel to LD, Stein variational gradient descent (SVGD) similarly minimizes the KL, albeit endowed with a novel Stein-Wasserstein distance, by deterministically transporting a set of particle samples, thus de-randomizes the stochastic diffusion process. We propose de-randomized kernel-based particle samplers to all diffusion-based samplers known as MCMC dynamics. Following previous work in interpreting MCMC dynamics, we equip the Stein-Wasserstein space with a fiber-Riemannian Poisson structure, with the capacity of characterizing a fiber-gradient Hamiltonian flow that simulates MCMC dynamics. Such dynamics discretizes into generalized SVGD (GSVGD), a Stein-type deterministic particle sampler, with particle updates coinciding with applying the diffusion Stein operator to a kernel function. We demonstrate empirically that GSVGD can de-randomize complex MCMC dynamics, which combine the advantages of auxiliary momentum variables and Riemannian structure, while maintaining the high sample quality from an interacting particle system.
Original language | English |
---|---|
Title of host publication | Advances in Neural Information Processing Systems 34 - 35th Conference on Neural Information Processing Systems, NeurIPS 2021 |
Publisher | Neural Information Processing Systems Foundation |
Pages | 17507-17517 |
Number of pages | 11 |
ISBN (Electronic) | 9781713845393 |
Publication status | Published - 2021 |
MoE publication type | A4 Article in a conference publication |
Event | Conference on Neural Information Processing Systems - Virtual, Online Duration: 6 Dec 2021 → 14 Dec 2021 Conference number: 35 https://neurips.cc |
Publication series
Name | Advances in Neural Information Processing Systems |
---|---|
Publisher | Neural Information Processing Systems Foundation |
Volume | 21 |
ISSN (Print) | 1049-5258 |
Conference
Conference | Conference on Neural Information Processing Systems |
---|---|
Abbreviated title | NeurIPS |
City | Virtual, Online |
Period | 06/12/2021 → 14/12/2021 |
Internet address |
Fingerprint
Dive into the research topics of 'De-randomizing MCMC dynamics with the diffusion Stein operator'. Together they form a unique fingerprint.-
Heinonen Markus AT-palkka: Deep learning with differential equations
01/09/2020 → 31/08/2025
Project: Academy of Finland: Other research funding
-
FCAI: Finnish Center for Artificial Intelligence
01/01/2019 → 31/12/2022
Project: Academy of Finland: Other research funding
-
Interactive machine learning from multiple biodata sources
Kaski, S., Hämäläinen, A., Gadd, C., Hegde, P., Shen, Z., Siren, J., Trinh, T., Jain, A. & Jälkö, J.
01/01/2019 → 31/08/2021
Project: Academy of Finland: Other research funding