Projects per year
Abstract
Approximate Bayesian inference estimates descriptors of an intractable target distribution - in essence, an optimization problem within a family of distributions. For example, Langevin dynamics (LD) extracts asymptotically exact samples from a diffusion process because the time evolution of its marginal distributions constitutes a curve that minimizes the KL-divergence via steepest descent in the Wasserstein space. Parallel to LD, Stein variational gradient descent (SVGD) similarly minimizes the KL, albeit endowed with a novel Stein-Wasserstein distance, by deterministically transporting a set of particle samples, thus de-randomizes the stochastic diffusion process. We propose de-randomized kernel-based particle samplers to all diffusion-based samplers known as MCMC dynamics. Following previous work in interpreting MCMC dynamics, we equip the Stein-Wasserstein space with a fiber-Riemannian Poisson structure, with the capacity of characterizing a fiber-gradient Hamiltonian flow that simulates MCMC dynamics. Such dynamics discretizes into generalized SVGD (GSVGD), a Stein-type deterministic particle sampler, with particle updates coinciding with applying the diffusion Stein operator to a kernel function. We demonstrate empirically that GSVGD can de-randomize complex MCMC dynamics, which combine the advantages of auxiliary momentum variables and Riemannian structure, while maintaining the high sample quality from an interacting particle system.
Original language | English |
---|---|
Title of host publication | Advances in Neural Information Processing Systems 34 pre-proceedings (NeurIPS 2021) |
Number of pages | 11 |
Publication status | Published - 2021 |
MoE publication type | A4 Article in a conference publication |
Event | Conference on Neural Information Processing Systems - Virtual, Online Duration: 6 Dec 2021 → 14 Dec 2021 Conference number: 35 https://neurips.cc |
Publication series
Name | Advances in Neural Information Processing Systems |
---|---|
Publisher | Morgan Kaufmann Publishers |
ISSN (Print) | 1049-5258 |
Conference
Conference | Conference on Neural Information Processing Systems |
---|---|
Abbreviated title | NeurIPS |
City | Virtual, Online |
Period | 06/12/2021 → 14/12/2021 |
Internet address |
Fingerprint
Dive into the research topics of 'De-randomizing MCMC dynamics with the diffusion Stein operator'. Together they form a unique fingerprint.-
Deep learning with differential equations
01/09/2020 → 31/08/2025
Project: Academy of Finland: Other research funding
-
FCAI: Finnish Center for Artificial Intelligence
01/01/2019 → 31/12/2022
Project: Academy of Finland: Other research funding
-
Interactive machine learning from multiple biodata sources
Jälkö, J., Hegde, P., Kaski, S., Gadd, C., Jain, A., Hämäläinen, A., Siren, J., Shen, Z. & Trinh, T.
01/01/2019 → 31/08/2021
Project: Academy of Finland: Other research funding