Robust, Accurate Stochastic Optimization for Variational Inference

Akash Dhaka, Alejandro Catalina Feliu, Michael Andersen, Måns Magnusson, Jonathan Huggins, Aki Vehtari

Tutkimustuotos: Artikkeli kirjassa/konferenssijulkaisussaConference contributionScientificvertaisarvioitu

Abstrakti

We consider the problem of fitting variational posterior approximations using
stochastic optimization methods. The performance of these approximations depends on (1) how well the variational family matches the true posterior distribution, (2) the choice of divergence, and (3) the optimization of the variational objective. We show that even in the best-case scenario when the exact posterior belongs to the assumed variational family, common stochastic optimization methods lead to poor variational approximations if the problem dimension is moderately large. We also demonstrate that these methods are not robust across diverse model types. Motivated by these findings, we develop a more robust and accurate stochastic optimization framework by viewing the underlying optimization algorithm as producing a Markov chain. Our approach is theoretically motivated and includes a diagnostic for convergence and a novel stopping rule, both of which are robust to
noisy evaluations of the objective function. We show empirically that the proposed framework works well on a diverse set of models: it can automatically detect stochastic optimization failure or inaccurate variational approximation.
AlkuperäiskieliEnglanti
OtsikkoThirty-fourth Conference on Neural Information Processing Systems
Sivumäärä12
TilaJulkaistu - 2020
OKM-julkaisutyyppiA4 Artikkeli konferenssijulkaisuussa
TapahtumaIEEE Conference on Neural Information Processing Systems; - Virtual, Vancouver, Kanada
Kesto: 6 joulukuuta 202012 joulukuuta 2020
Konferenssinumero: 34

Julkaisusarja

NimiAdvances in Neural Information Processing Systems
KustantajaMORGAN KAUFMANN PUBLISHERS
Vuosikerta33
ISSN (elektroninen)1049-5258

Conference

ConferenceIEEE Conference on Neural Information Processing Systems;
LyhennettäNeurIPS
MaaKanada
KaupunkiVancouver
Ajanjakso06/12/202012/12/2020

Sormenjälki Sukella tutkimusaiheisiin 'Robust, Accurate Stochastic Optimization for Variational Inference'. Ne muodostavat yhdessä ainutlaatuisen sormenjäljen.

Siteeraa tätä