HackerPapers

arXiv:1606.05908stat.ML

Tutorial on Variational Autoencoders

Carl Doersch

Abstract

In just three years, Variational Autoencoders (VAEs) have emerged as one of the most popular approaches to unsupervised learning of complicated distributions. VAEs are appealing because they are built on top of standard function approximators (neural networks), and can be trained with stochastic gradient descent. VAEs have already shown promise in generating many kinds of complicated data, including handwritten digits, faces, house numbers, CIFAR images, physical models of scenes, segmentation, and predicting the future from static images. This tutorial introduces the intuitions behind VAEs, explains the mathematics behind them, and describes some empirical behavior. No prior knowledge of variational Bayesian methods is assumed.

arXiv comments
Journal reference
DOI
Subjects
stat.MLMachine Learning (Statistics) cs.LGMachine Learning

On Hacker News

Linked by 2 identifiable commenters in 2 comments across 2 threads, from Jul 2017 to Mar 2023.

Comments linking this paper, per year.

Discussed in comments2 threads

  1. Introduction to Autoencoders 1 comment

    • To grok the ELBO, don't start with VAEs; start with basic variational inference. https://arxiv.org/abs/1601.00670 is good, though maybe increasingly dated...Once you're happy with VI in general, https://arxiv.org/abs/1606.05908 is a nice read on the VAE.

      cgs1019View on HNParent context

  2. Hacker's guide to Neural Networks (2012) 1 comment

    • No, stochastic gradient descent (SGD) needs a differentiable loss function (differentiable with respect to the trainable parameters). The stochastic bit is that you want to optimize the loss for the whole dataset but what you actually do is to do each gradient descent step on a…

      albertzeyerView on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.