Denoising autoencoder with modulated lateral connections learns invariant representations of natural images

Antti Rasmus, Tapani Raiko, Harri Valpola

Tutkimustuotos: Artikkeli kirjassa/konferenssijulkaisussaKonferenssiesitysScientificvertaisarvioitu

Abstrakti

Suitable lateral connections between encoder and decoder are shown to allow higher layers of a denoising autoencoder (dAE) to focus on invariant representations. In regular autoencoders, detailed information needs to be carried through the highest layers but lateral connections from encoder to decoder relieve this pressure. It is shown that abstract invariant features can be translated to detailed reconstructions when invariant features are allowed to modulate the strength of the lateral connection. Three dAE structures with modulated and additive lateral connections, and without lateral connections were compared in experiments using real-world images. The experiments verify that adding modulated lateral connections to the model 1) improves the accuracy of the probability model for inputs, as measured by denoising performance; 2) results in representations whose degree of invariance grows faster towards the higher layers; and 3) supports the formation of diverse invariant poolings.

AlkuperäiskieliEnglanti
TilaJulkaistu - 1 tammikuuta 2015
OKM-julkaisutyyppiEi oikeutettu
TapahtumaInternational Conference on Learning Representations - San Diego, Yhdysvallat
Kesto: 7 toukokuuta 20159 toukokuuta 2015
Konferenssinumero: 3

Conference

ConferenceInternational Conference on Learning Representations
LyhennettäICLR
MaaYhdysvallat
KaupunkiSan Diego
Ajanjakso07/05/201509/05/2015

Sormenjälki Sukella tutkimusaiheisiin 'Denoising autoencoder with modulated lateral connections learns invariant representations of natural images'. Ne muodostavat yhdessä ainutlaatuisen sormenjäljen.

  • Siteeraa tätä

    Rasmus, A., Raiko, T., & Valpola, H. (2015). Denoising autoencoder with modulated lateral connections learns invariant representations of natural images. Julkaisun esittämispaikka: International Conference on Learning Representations, San Diego, Yhdysvallat.