Improving Variational Inference with Inverse Autoregressive Flow · HackerPapers discussions

Improving Variational Inference with Inverse Autoregressive Flow — mentioned by taliesinb

Tue, 16 Aug 2016 10:23:06 GMT

VAEs can be extended to make the latent variables dependent. OpenAI's inverse autoregressive flow is one recent way that is particularly efficient: http://arxiv.org/pdf/1606.04934v1.pdf. Linear IAF is the simplest form of this, with it you can model normal z having an arbitrary covariance matrix. But aside from that, there is an information-theoretic view on why you might prefer VAEs over AEs. In short, having p(z|x) not be point-mass (aka an ordinary AE) allows you to bound the information flow through the bottleneck. KL loss on p(z|x) forces the network to be honest about how much…

Paper and discussion history. A mention is not an endorsement.

Improving Variational Inference with Inverse Autoregressive Flow — mentioned by andreyk

Thu, 16 Jun 2016 20:51:48 GMT

Brief summary: a nice intro about what generative models are and the current popular approaches/papers, followed by descriptions of recent work by OpenAI in the space. Quick links to papers mentioned: Improving GANs https://arxiv.org/abs/1606.03498 Improving VAEs http://arxiv.org/abs/1606.04934 InfoGAN https://arxiv.org/abs/1606.03657 Curiosity-driven Exploration in Deep Reinforcement Learning via Bayesian Neural Networks http://arxiv.org/abs/1605.09674 Generative Adversarial Imitation Learning http://arxiv.org/abs/1606.03476 I think the last one seems very exciting, I expect Imitation…

Paper and discussion history. A mention is not an endorsement.