HackerPapers

arXiv:1612.00410cs.LG

Deep Variational Information Bottleneck

Alexander A. Alemi, Ian Fischer, Joshua V. Dillon, Kevin Murphy

Early discovery. Linked by one identifiable commenter. This paper is outside the main ranking; a mention is not an endorsement.

Abstract

We present a variational approximation to the information bottleneck of Tishby et al. (1999). This variational approach allows us to parameterize the information bottleneck model using a neural network and leverage the reparameterization trick for efficient training. We call this method "Deep Variational Information Bottleneck", or Deep VIB. We show that models trained with the VIB objective outperform those that are trained with other forms of regularization, in terms of generalization performance and robustness to adversarial attack.

arXiv comments
19 pages, 8 figures, Accepted to ICLR17
Journal reference
Proceedings of the International Conference on Learning Representations (ICLR) 2017
DOI
Subjects
cs.LGMachine Learning cs.ITInformation Theory

On Hacker News

Linked by 1 identifiable commenter in 1 comment across 1 thread, from Sep 2018 to Sep 2018.

Comments linking this paper, per year.

Discussed in comments1 thread

  1. Variational Autoencoders Explained 1 comment

    • Two things I believe to be true... 1) Auto-encoders are overplayed, mostly because they're a pretty easy intro ML project. There was a brief moment (before ResNets and batch normalization showed up) when they were useful for bootstrapping a representation, but they aren't…

      sdenton4View on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.