HackerPapers

arXiv:2006.11239cs.LG

Denoising Diffusion Probabilistic Models

Jonathan Ho, Ajay Jain, Pieter Abbeel

Abstract

We present high quality image synthesis results using diffusion probabilistic models, a class of latent variable models inspired by considerations from nonequilibrium thermodynamics. Our best results are obtained by training on a weighted variational bound designed according to a novel connection between diffusion probabilistic models and denoising score matching with Langevin dynamics, and our models naturally admit a progressive lossy decompression scheme that can be interpreted as a generalization of autoregressive decoding. On the unconditional CIFAR10 dataset, we obtain an Inception score of 9.46 and a state-of-the-art FID score of 3.17. On 256x256 LSUN, we obtain sample quality similar to ProgressiveGAN. Our implementation is available at https://github.com/hojonathanho/diffusion

arXiv comments
Journal reference
DOI
Subjects
cs.LGMachine Learning stat.MLMachine Learning (Statistics)

On Hacker News

Linked by 5 identifiable commenters in 6 comments across 6 threads, from Sep 2022 to May 2024.

Comments linking this paper, per year.

Discussed in comments6 threads

  1. Diffusion Models 1 comment

    • The original Dickstein 2015 paper [1] formulated diffusion as maximizing (a lower bound of) log-likelihood of generating the distribution, so there was some theory. But my understanding is that the breakthrough was empirical results from Ho [2] and Nichol [3] showing diffusion…

      reasonablekloutView on HNParent context

  2. Mamba Explained: The State Space Model Taking On Transformers 1 comment

    • Yes and no. The thing is that Mamba is not perfect. There's no neural architecture to rule them all, if you will. I think the bigger issue is that we more act like there is and get on bandwagons. Let me give a clearer example from the past so we can see. The predecessor to DDPM…

      godelskiView on HNParent context

  3. C.R. Rao: A Life in Statistics (2020) 1 comment
  4. Stanford A.I. Courses 1 comment

    • Yeah no problem, this is even closer to my area of focus! What do you know about physics and thermodynamics? I'd say a good intro for low background is from Tomczak[0]. He has a book, but the blog posts are nearly identical. He did a post doc with Max Welling (someone you should…

      godelskiView on HNParent context

  5. We’ve filed a law­suit chal­leng­ing Sta­ble Dif­fu­sion 1 comment

    • Yeah, and their next figure isn't any better. They show a latent space interpolation figure from DDPM, and they seem to think this is how Diffusion models produce a "collage" (as they describe the process). Of course, this figure has nothing to do with how image generation is…

      ImnimoView on HNParent context

  6. Stable Diffusion forming images from text: image snapshots at each step 1 comment

    • It is fed using a fascinating mechanism called "cross-attention" that originated in the NN architecture called Transformer - which was used to achieve state-of-the-art translations. It uses something like associative memory, where a NN inside Stable Diffusion, that generates…

      DzugaruView on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.