HackerPapers

arXiv:2404.02905cs.CV

Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction

Keyu Tian, Yi Jiang, Zehuan Yuan, Bingyue Peng, Liwei Wang

Abstract

We present Visual AutoRegressive modeling (VAR), a new generation paradigm that redefines the autoregressive learning on images as coarse-to-fine "next-scale prediction" or "next-resolution prediction", diverging from the standard raster-scan "next-token prediction". This simple, intuitive methodology allows autoregressive (AR) transformers to learn visual distributions fast and generalize well: VAR, for the first time, makes GPT-like AR models surpass diffusion transformers in image generation. On ImageNet 256x256 benchmark, VAR significantly improve AR baseline by improving Frechet inception distance (FID) from 18.65 to 1.73, inception score (IS) from 80.4 to 350.2, with around 20x faster inference speed. It is also empirically verified that VAR outperforms the Diffusion Transformer (DiT) in multiple dimensions including image quality, inference speed, data efficiency, and scalability. Scaling up VAR models exhibits clear power-law scaling laws similar to those observed in LLMs, with linear correlation coefficients near -0.998 as solid evidence. VAR further showcases zero-shot generalization ability in downstream tasks including image in-painting, out-painting, and editing. These results suggest VAR has initially emulated the two important properties of LLMs: Scaling Laws and zero-shot task generalization. We have released all models and codes to promote the exploration of AR/VAR models for visual generation and unified learning.

arXiv comments
Demo website: https://var.vision/
Journal reference
DOI
Subjects
cs.CVComputer Vision and Pattern Recognition cs.AIArtificial Intelligence

On Hacker News

Linked by 4 identifiable commenters in 8 comments across 5 threads, from Oct 2024 to Aug 2025. Found in 2 HN submissions from Apr 2024 to Oct 2024. Open submission discussions.

Comments linking this paper, per year.

Discussed in comments5 threads

  1. Ask HN: Who wants to be hired? (August 2025) 1 comment

    • Location: Southern California Remote: Yes Willing to relocate: Yes Technologies: C++, Python, Hugging Face Resume: Can share on request Email: peterpeternews@gmail.com I recently graduated a PhD in theoretical computer science. I have a strong mathematical background and…

      peter_peterView on HNParent context

  2. Diffusion models explained simply 1 comment

    • It's nice that this contains a comparison between diffusion models that are used for image models, and the autoregressive models that are used for LLMs. But recently (2024 NeuIPS paper of the year) there was a new paper on autoregressive image modelling that apparently…

      cubefoxView on HNParent context

  3. No elephants: Breakthroughs in image generation 4 comments

    • Open AI have both said it's native image generation and autoregressive. It has the signs of it too. It's probably an implementation of VAR (https://arxiv.org/abs/2404.02905) - autoregressive image generation with a small twist. Rather than predict every token at the target…

      famouswafflesView on HNParent context

    • It's kind of clear that for every request, it generates a new image entirely. Some people are speculating a diffusion decoder but i think it's more likely an implementation of VAR - https://arxiv.org/abs/2404.02905. So rather than predicting each patch at the target resolution…

      famouswafflesView on HNParent context

    • I mean, it could very well be that it generates image patches autoregressively, but in a pyramidal way (first a very low resolution version, the "canvas", and then each individual patch). This is very similar to VAR [1] We can't really be sure until OpenAI tells us. [1]…

      andy12_View on HNParent context

    • This video is very good. https://youtu.be/EzDsrEvdgNQ?si=EWp3U1GMkwg1bMQQ One thing i'd add is that generating the tokens at the target resolution from the start is no longer the only approach to autoregressive image generation. Rather than predicting each patch at the target…

      famouswafflesView on HNParent context

  4. 4o Image Generation 1 comment

    • There is a more recent approach to auto-regressive image generation. Rather than predicting the next patch at the target resolution one by one, it predicts the next resolution. That is, the image at a small resolution followed by the image at a higher resolution and so on.…

      famouswafflesView on HNParent context

  5. Faster convergence for diffusion models 1 comment

    • The most popular method using autoregression in image generation space is to predict image patches/tokens and not pixels, though that still scales worse than diffusion. A fairly new but promising approach for autoregression that seems to scale as well as diffusion is predicting…

      famouswafflesView on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.