HackerPapers

arXiv:2310.04378cs.CV

Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference

Simian Luo, Yiqin Tan, Longbo Huang, Jian Li, Hang Zhao

Abstract

Latent Diffusion models (LDMs) have achieved remarkable results in synthesizing high-resolution images. However, the iterative sampling process is computationally intensive and leads to slow generation. Inspired by Consistency Models (song et al.), we propose Latent Consistency Models (LCMs), enabling swift inference with minimal steps on any pre-trained LDMs, including Stable Diffusion (rombach et al). Viewing the guided reverse diffusion process as solving an augmented probability flow ODE (PF-ODE), LCMs are designed to directly predict the solution of such ODE in latent space, mitigating the need for numerous iterations and allowing rapid, high-fidelity sampling. Efficiently distilled from pre-trained classifier-free guided diffusion models, a high-quality 768 x 768 2~4-step LCM takes only 32 A100 GPU hours for training. Furthermore, we introduce Latent Consistency Fine-tuning (LCF), a novel method that is tailored for fine-tuning LCMs on customized image datasets. Evaluation on the LAION-5B-Aesthetics dataset demonstrates that LCMs achieve state-of-the-art text-to-image generation performance with few-step inference. Project Page: https://latent-consistency-models.github.io/

arXiv comments
Journal reference
DOI
Subjects
cs.CVComputer Vision and Pattern Recognition cs.LGMachine Learning

On Hacker News

Linked by 3 identifiable commenters in 3 comments across 1 thread, from Nov 2023 to Nov 2023.

Comments linking this paper, per year.

Discussed in comments1 thread

  1. Real-time image editing using latent consistency models 3 comments

    • it uses a new technique called "consistency" that lets latent diffusion models to predict images in much fewer steps. some links here: - https://arxiv.org/abs/2310.04378 - https://arxiv.org/abs/2311.05556

      vipermuView on HNParent context

    • from https://arxiv.org/abs/2310.04378 it sounds like its a form of distillation of an SDL model. So im guessing it can't be directly trained, but once you have a trained diffusion model you can distil a predictor which cuts out the iterative steps. While it can do 1-step the…

      quadratureView on HNParent context

    • Let's quote the whole sentence, shall we? An LCM demands merely 32 A100 GPUs Hours training for 2-step inference, as de- picted in Figure 1. [1] Now let's look at the caption under Figure 1: LCMs can be distilled from any pre-trained Stable Diffusion (SD) in only 4,000 training…

      T-AView on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.