MaskGIT: Masked Generative Image Transformer — mentioned by thomasahle

Thu, 22 May 2025 19:30:29 GMT

> Now, diffusion LMs take this idea further. BERT can recover 15% of masked tokens ("noise"), but why stop here. Let's train a model to recover texts with 30%, 50%, 90%, 100% of masked tokens. > Once you've trained that, in order to generate something from scratch, you start by feeding the model all [MASK]s. It will generate you mostly gibberish, but you can take some tokens (let's say, 10%) at random positions and assume that these tokens are generated ("final") This is clearly wrong. If you actually froze 10% of gibberish tokens, your output would be terrible! What you actually do in…

Paper and discussion history. A mention is not an endorsement.

MaskGIT: Masked Generative Image Transformer · HackerPapers discussions

MaskGIT: Masked Generative Image Transformer — mentioned by thomasahle