Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Model · HackerPapers discussions

Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Model — mentioned by thegeomaster

Fri, 25 Apr 2025 20:56:26 GMT

It's a mix between the Transformer architecture and diffusion, shown to provide better output results than simple autoregressive image token generation alone: https://arxiv.org/html/2408.11039v1 Of course, nobody really knows what 4o image generation really is under the hood, but it looks to be like some kind of hybrid system like Transfusion to me. It is much better at prompt adherence than diffusion models, but its output can be clunkier/stylistically incoherent. At times, it also exhibits similar failure modes as diffusion (such as weirdly rotated body parts). Given how it behaves, I think…

Paper and discussion history. A mention is not an endorsement.

Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Model — mentioned by sva_

Wed, 25 Sep 2024 21:33:34 GMT

Curious about the multimodal model's architecture. But alas, when I try to request access > Llama 3.2 Multimodal is not available in your region. It sounds like they input the continuous output of an image encoder into a transformer, similar to transfusion[0]? Does someone know where to find more details? Edit: > Regarding the licensing terms, Llama 3.2 comes with a very similar license to Llama 3.1, with one key difference in the acceptable use policy: any individual domiciled in, or a company with a principal place of business in, the European Union is not being granted the license rights…

Paper and discussion history. A mention is not an endorsement.

Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Model — mentioned by cosmicjedi

Mon, 09 Sep 2024 22:49:38 GMT

You can talk to the authors directly on alphaXiv! https://www.alphaxiv.org/abs/2408.11039v1

Paper and discussion history. A mention is not an endorsement.

Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Model — HN submission

Mon, 09 Sep 2024 18:51:31 GMT

Transfusion: Predict the next token and diffuse images with one multimodal model

122 points; 10 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Model — HN submission

Wed, 21 Aug 2024 14:29:45 GMT

Predict the Next Token and Diffuse Images with One Multi-Modal Model

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.