Ladder Variational Autoencoders

Casper Kaae Sønderby, Tapani Raiko, Lars Maaløe, Søren Kaae Sønderby, Ole Winther

Tutkimustuotos: Artikkeli kirjassa/konferenssijulkaisussaConference contributionScientificvertaisarvioitu

Abstrakti

Variational autoencoders are powerful models for unsupervised learning. However deep models with several layers of dependent stochastic variables are difficult to train which limits the improvements obtained using these highly expressive models. We propose a new inference model, the Ladder Variational Autoencoder, that recursively corrects the generative distribution by a data dependent approximate likelihood in a process resembling the recently proposed Ladder Network. We show that this model provides state of the art predictive log-likelihood and tighter log-likelihood lower bound compared to the purely bottom-up inference in layered Variational Autoencoders and other generative models. We provide a detailed analysis of the learned hierarchical latent representation and show that our new inference model is qualitatively different and utilizes a deeper more distributed hierarchy of latent variables. Finally, we observe that batch-normalization and deterministic warm-up (gradually turning on the KL-term) are crucial for training variational models with many stochastic layers.
AlkuperäiskieliEnglanti
OtsikkoAdvances in Neural Information Processing Systems
KustantajaNeural Information Processing Systems Foundation
Sivut3745-3753
Sivumäärä9
TilaJulkaistu - 2016
OKM-julkaisutyyppiA4 Artikkeli konferenssijulkaisuussa
TapahtumaIEEE Conference on Neural Information Processing Systems - Barcelona, Espanja
Kesto: 5 joulukuuta 201610 joulukuuta 2016
Konferenssinumero: 30

Julkaisusarja

NimiAdvances in neural information processing systems
KustantajaNeural Information Processing Systems Foundation
Vuosikerta29
ISSN (painettu)1049-5258

Conference

ConferenceIEEE Conference on Neural Information Processing Systems
LyhennettäNIPS
MaaEspanja
KaupunkiBarcelona
Ajanjakso05/12/201610/12/2016

Sormenjälki Sukella tutkimusaiheisiin 'Ladder Variational Autoencoders'. Ne muodostavat yhdessä ainutlaatuisen sormenjäljen.

  • Siteeraa tätä

    Kaae Sønderby, C., Raiko, T., Maaløe, L., Kaae Sønderby, S., & Winther, O. (2016). Ladder Variational Autoencoders. teoksessa Advances in Neural Information Processing Systems (Sivut 3745-3753). (Advances in neural information processing systems; Vuosikerta 29). Neural Information Processing Systems Foundation. http://papers.nips.cc/paper/6275-ladder-variational-autoencoders