Denoising Diffusion Probabilistic Models · HackerPapers discussions

Denoising Diffusion Probabilistic Models — mentioned by reasonableklout

Sun, 26 May 2024 21:33:14 GMT

The original Dickstein 2015 paper [1] formulated diffusion as maximizing (a lower bound of) log-likelihood of generating the distribution, so there was some theory. But my understanding is that the breakthrough was empirical results from Ho [2] and Nichol [3] showing diffusion could produce not only high-quality samples but better than GANs in some cases. [1] https://arxiv.org/abs/1503.03585 [2] https://arxiv.org/abs/2006.11239 [3] https://arxiv.org/abs/2105.05233

Paper and discussion history. A mention is not an endorsement.

Denoising Diffusion Probabilistic Models — mentioned by godelski

Sun, 25 Feb 2024 18:59:25 GMT

Yes and no. The thing is that Mamba is not perfect. There's no neural architecture to rule them all, if you will. I think the bigger issue is that we more act like there is and get on bandwagons. Let me give a clearer example from the past so we can see. The predecessor to DDPM (the work that kicked off the diffusion model era) was published in 2015[0], only a year after GANs[1]. Diffusion then showed promise but why did DDPM come out in 2020[2]? Because everyone was working on GANs. GANs produced far better images and diffusion (still is) was a lot more computationally intensive. Plus, all…

Paper and discussion history. A mention is not an endorsement.

Denoising Diffusion Probabilistic Models — mentioned by PartiallyTyped

Tue, 29 Aug 2023 15:23:18 GMT

Also diffusion models [1,2]. [1] https://en.wikipedia.org/wiki/Rao%E2%80%93Blackwell_theorem [2] https://arxiv.org/abs/2006.11239

Paper and discussion history. A mention is not an endorsement.

Denoising Diffusion Probabilistic Models — mentioned by godelski

Sun, 02 Jul 2023 23:10:11 GMT

Yeah no problem, this is even closer to my area of focus! What do you know about physics and thermodynamics? I'd say a good intro for low background is from Tomczak[0]. He has a book, but the blog posts are nearly identical. He did a post doc with Max Welling (someone you should learn about if you want to get deep, like I was suggesting before). So I'd switch things up slightly. I'd go Intro -> Autoregressive -> Flow -> VAE -> Hierarchical VAE -> Energy Based Models -> Diffusion. It is worth learning about GANs btw, but this progression should be natural and build up. Continuing from there…

Paper and discussion history. A mention is not an endorsement.

Denoising Diffusion Probabilistic Models — mentioned by Imnimo

Sat, 14 Jan 2023 17:55:06 GMT

Yeah, and their next figure isn't any better. They show a latent space interpolation figure from DDPM, and they seem to think this is how Diffusion models produce a "collage" (as they describe the process). Of course, this figure has nothing to do with how image generation is actually performed. It's just an experiment for the purpose of the paper to demonstrate that the latent space is structured. In fact, this only works because the source images are given as input to the forward process - thus, the details being interpolated are from the inputs not from the model. If you look at Appendix…

Paper and discussion history. A mention is not an endorsement.

Denoising Diffusion Probabilistic Models — mentioned by Dzugaru

Sat, 03 Sep 2022 09:29:51 GMT

It is fed using a fascinating mechanism called "cross-attention" that originated in the NN architecture called Transformer - which was used to achieve state-of-the-art translations. It uses something like associative memory, where a NN inside Stable Diffusion, that generates image (UNet, working in latent space), at almost each step "asks" the whole encoded prompt to provide data at various positions using query vector Q that is matched against key vectors K and value vectors V [0]. How Stable Diffusion works [1] as a whole is not really hard to comprehend at a high level - but you'll need…

Paper and discussion history. A mention is not an endorsement.