HackerPapers

arXiv:2102.12092cs.CV

Zero-Shot Text-to-Image Generation

Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, Ilya Sutskever

Abstract

Text-to-image generation has traditionally focused on finding better modeling assumptions for training on a fixed dataset. These assumptions might involve complex architectures, auxiliary losses, or side information such as object part labels or segmentation masks supplied during training. We describe a simple approach for this task based on a transformer that autoregressively models the text and image tokens as a single stream of data. With sufficient data and scale, our approach is competitive with previous domain-specific models when evaluated in a zero-shot fashion.

arXiv comments
Journal reference
DOI
Subjects
cs.CVComputer Vision and Pattern Recognition cs.LGMachine Learning

On Hacker News

Linked by 5 identifiable commenters in 6 comments across 6 threads, from Feb 2021 to Feb 2023. Found in 1 HN submission from Feb 2021 to Feb 2021. Open submission discussions.

Comments linking this paper, per year.

Discussed in comments6 threads

  1. The Norvig – Chomsky debate (2017) 1 comment

    • DALL-E 1 used a GPT approach to generate both text and images. Images are divided into patches, about 1024 patches for one image. Each patch is like a text token. > We describe a simple approach for this task based on a transformer that autoregressively models the text and image…

      visargaView on HNParent context

  2. DALL-E 2 has a secret language 1 comment

    • If they use BPE dropout, then the split can be different and not unique. And for the record, they use BPE dropout for DALLE-1, see https://arxiv.org/pdf/2102.12092.pdf

      karmasimidaView on HNParent context

  3. DALL-E, the Metaverse, and Zero Marginal Content 1 comment
  4. Horse-riding astronaut is a milestone in AI’s journey to make sense of the world 1 comment

    • Their research remains open: https://arxiv.org/abs/2102.12092 Their code/models are indeed closed, but there is no realistic alternative. If they let the public have unrestricted access, deepfakes + child images would appear on Day 1, and OpenAI would get cancelled. For OpenAI…

      exo-pla-netView on HNParent context

  5. Wit: Wikipedia-Based Image Text Dataset 1 comment

    • DALL-E is 250m: https://arxiv.org/pdf/2102.12092.pdf#page=14 (so 20x larger). CLIP (https://cdn.openai.com/papers/Learning_Transferable_Visual_M...) is 400m (34x larger). And ALIGN (https://arxiv.org/abs/2102.05918#google) is 1800m (155x). (For 47m maybe you're thinking of a…

      gwernView on HNParent context

  6. DALL-E Paper and Code 1 comment

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.