Tutorial on Variational Autoencoders — mentioned by cgs1019

Wed, 29 Mar 2023 15:30:01 GMT

To grok the ELBO, don't start with VAEs; start with basic variational inference. https://arxiv.org/abs/1601.00670 is good, though maybe increasingly dated...Once you're happy with VI in general, https://arxiv.org/abs/1606.05908 is a nice read on the VAE.

Paper and discussion history. A mention is not an endorsement.

Tutorial on Variational Autoencoders — mentioned by albertzeyer

Fri, 14 Jul 2017 17:27:56 GMT

No, stochastic gradient descent (SGD) needs a differentiable loss function (differentiable with respect to the trainable parameters). The stochastic bit is that you want to optimize the loss for the whole dataset but what you actually do is to do each gradient descent step on a small mini-batch of the dataset. There might be extensions where you have some non-differentiable part where you just assume some gradient in back-propagation. Like Gumble-Softmax (https://arxiv.org/abs/1611.01144). Or similar is the reparametrization trick in VAEs (https://arxiv.org/abs/1606.05908). But that are…

Paper and discussion history. A mention is not an endorsement.

Tutorial on Variational Autoencoders · HackerPapers discussions

Tutorial on Variational Autoencoders — mentioned by cgs1019

Tutorial on Variational Autoencoders — mentioned by albertzeyer