Scalable Diffusion Models with Transformers · HackerPapers discussions

Scalable Diffusion Models with Transformers — mentioned by godelski

Sun, 29 Jun 2025 20:59:47 GMT

It's a bit more complicated than that and I don't think you're being fair. StarFlow and the AR models are fixed but DiT is being compared at different amount of steps and we don't really care if we generate garbage at blazing speeds[0]. Go look at... also Figure 10 (lol) from the DiT paper[1], it compares FID to model sizes and sampling steps. It looks like StarFlow is comparing to DiT-XL/2-G. In [1] they do {16,32,64,128,256,1024} steps which corresponds to (roughly) 10k-FID of 60, 35, 25, 22, 21, 20. Translating to StarFlow's graph we'll guesstimate 21,23,50. There's a big difference…

Paper and discussion history. A mention is not an endorsement.

Scalable Diffusion Models with Transformers — mentioned by cubefox

Thu, 22 May 2025 01:56:34 GMT

Image diffusion models also use transformers nowadays. Here is the original "diffusion transformer" paper: https://arxiv.org/abs/2212.09748 Earlier image diffusion models used U-nets: https://en.wikipedia.org/wiki/U-Net

Paper and discussion history. A mention is not an endorsement.

Scalable Diffusion Models with Transformers — mentioned by albertzeyer

Thu, 22 Feb 2024 17:47:38 GMT

I understand that Sora is very popular, so it makes sense to refer to it, but when saying it is similar to Sora, I guess it actually makes more sense to say that it uses a Diffusion Transformer (DiT) (https://arxiv.org/abs/2212.09748) like Sora. We don't really know more details on Sora, while the original DiT has all the details.

Paper and discussion history. A mention is not an endorsement.

Scalable Diffusion Models with Transformers — mentioned by dr_dshiv

Fri, 16 Feb 2024 08:31:34 GMT

On the other hand, this is the first convincing use of a “diffusion transformer” [1]. My understanding is that videos and images are tokenized into patches, through a process that compresses the video/images into abstracted concepts in latent space. Those patches (image/video concepts in latent space) can then be used with transformers (because patches are the tokens). The point is that there is plenty of room for optimization following the first demonstration of a new architecture. Edit: sorry, it’s not the first diffusion transformer. That would be [2] [1]…

Paper and discussion history. A mention is not an endorsement.

Scalable Diffusion Models with Transformers — HN submission

Thu, 24 Aug 2023 11:45:56 GMT

Diffusion Transformer

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.