HackerPapers

arXiv:1605.08803cs.LG

Density estimation using Real NVP

Laurent Dinh, Jascha Sohl-Dickstein, Samy Bengio

Abstract

Unsupervised learning of probabilistic models is a central yet challenging problem in machine learning. Specifically, designing models with tractable learning, sampling, inference and evaluation is crucial in solving this task. We extend the space of such models using real-valued non-volume preserving (real NVP) transformations, a set of powerful invertible and learnable transformations, resulting in an unsupervised learning algorithm with exact log-likelihood computation, exact sampling, exact inference of latent variables, and an interpretable latent space. We demonstrate its ability to model natural images on four datasets through sampling, log-likelihood evaluation and latent variable manipulations.

arXiv comments
10 pages of main content, 3 pages of bibliography, 18 pages of appendix. Accepted at ICLR 2017
Journal reference
DOI
Subjects
cs.LGMachine Learning cs.AIArtificial Intelligence cs.NENeural and Evolutionary Computing stat.MLMachine Learning (Statistics)

On Hacker News

Linked by 3 identifiable commenters in 4 comments across 4 threads, from Jun 2016 to Apr 2020.

Comments linking this paper, per year.

Discussed in comments4 threads

  1. Making an invertible function out of non-invertible parts 1 comment

    • Another example is RealNVP, which has the same structure to enable inversion (partition, process half, then use the result to mutate the other half in an invertible way). Sadly, I gather the computational performance Is pretty bad .. https://arxiv.org/abs/1605.08803

      fvnccView on HNParent context

  2. Deep Learning State of the Art (2019) [video] 1 comment

    • Nice work. I can think of only two things that are missing: * Normalizing flows - e.g., https://arxiv.org/abs/1605.08803 , https://arxiv.org/abs/1807.03039 , among many others * ODEnets and continous normalizing flows - https://arxiv.org/abs/1806.07366

      cs702View on HNParent context

  3. Tale of Three Probabilistic Families: Discriminative, Descriptive and Generative 1 comment

    • These guys have done what looks like an impressive amount of work... but I was disappointed to see there's no mention of generative language modeling at all and only a very brief mention of key advances in image generation over the past few years. Examples of generative language…

      cs702View on HNParent context

  4. Blade Runner re-encoded using neural networks 1 comment

    • Yes, although the particular technique used here (variational autoencoder, or VAE) doesn't benefit from increasing the code space due to a particular penalty (KL divergence against a fixed N(0, 1) Gaussian prior) during training. So even by making it go to 1000 digits in the…

      kastnerkyleView on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.