Information bottlenecks and dimensionality reduction in deep learning: Notes on speech processing

Lehdistö/media: Sosiaalisen median aktiviteetti

Description

Autoencoders and other deep neural networks with information bottlenecks have become fashionable. The heuristic idea is that the dimensionality of the hidden layers is reduced such that the network is forced to focus on the important part of the data. Experiments have also demonstrated that autoencoders are efficient in this sense. I have however been left wondering whether the amount of information can be characterized in exact terms. How much information flows through the bottleneck? How would we even measure that?

Aikajakso3 syysk. 2020

Osallistuminen mediassa

1

Osallistuminen mediassa

  • NimiInformation bottlenecks and dimensionality reduction in deep learning: Notes on speech processing
    Medianimi / kanavaMedium
    MediatyyppiWeb
    Julkaisupäivämäärä03/09/2020
    KuvausAutoencoders and other deep neural networks with information bottlenecks have become fashionable. The heuristic idea is that the dimensionality of the hidden layers is reduced such that the network is forced to focus on the important part of the data. Experiments have also demonstrated that autoencoders are efficient in this sense. I have however been left wondering whether the amount of information can be characterized in exact terms. How much information flows through the bottleneck? How would we even measure that?
    URL-osoitehttps://towardsdatascience.com/information-bottlenecks-c2ee67015065
    HenkilötTom Bäckström
Näytä kaikki 1
Näytä vähemmän