Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding · HackerPapers discussions

Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding — mentioned by airgapstopgap

Wed, 26 Jul 2023 20:24:01 GMT

Diffusion is more parameter-efficient and you quickly saturate the target fidelity, especially with some refiner cascade. It's a solved problem. You do not need more than maybe 4B total. Images are far more redundant than text. In fact, most interesting papers since Imagen show that you get more mileage out of scaling the text encoder part, which is, of course, a Transformer. This is what drives accuracy, text rendering, compositionality, parsing edge cases. In SD 1.5 the text encoder part (CLIP ViT-L/14) takes a measly 123M parameters.[1] In Imagen, it was T5-XXL with 4.6B [2]. I am…

Paper and discussion history. A mention is not an endorsement.

Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding — mentioned by godelski

Mon, 22 May 2023 23:09:10 GMT

I definitely don't buy these papers at face value. I say this as an ML researcher btw. You'll often see these works discussing zero-shot performance. But many of these tasks are either not zero-shot or even a known n-shot. Let's take a good example, Imagen[0] claims zero-shot MS-COCO performance but trains on LAION. COCO classes exist in LAION and there are similar texts. Explore COCO[1] and explore clip retrieval[2] for LAION. The example given is the first sample from COCO aircraft and you'll find almost identical images and captions with many of the same keywords. This isn't zero-shot.…

Paper and discussion history. A mention is not an endorsement.

Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding — mentioned by FL33TW00D

Fri, 28 Apr 2023 17:23:22 GMT

Check out figure 4A from the ImageGen paper: https://arxiv.org/pdf/2205.11487.pdf From that - I would strongly suspect the answer to your question to be yes.

Paper and discussion history. A mention is not an endorsement.

Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding — mentioned by espadrine

Fri, 10 Mar 2023 08:28:17 GMT

Model sizes are adjusted to reach a given quality. Many papers justified empirically the use of more neurons allocated to language than to visual understanding, like Imagen[0] (figure 4.a). as it yields better results. Debatably, it is also true in the human brain. To me, it could highlight that there is more entropy in language than in images. After all, games have a similar property: some games, like noughts and crosses, or checkers, can have the same model architecture reach optimal play with smaller sizes than are necessary for chess and go[1]. It is certainly true that language has a lot…

Paper and discussion history. A mention is not an endorsement.

Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding — mentioned by wodenokoto

Thu, 25 Aug 2022 11:23:21 GMT

Googling the author list, gives me a preprint dated March this year. https://arxiv.org/abs/2205.11487

Paper and discussion history. A mention is not an endorsement.