Know, then Draw: A New Learning Paradigm for Visual Generation
What it is
This paper proposes a two-stage learning paradigm for visual generation that uses autoregressive pretraining for semantic knowledge acquisition and diffusion post-training for high-fidelity rendering. It introduces a purely semantic tokenizer trained with language supervision and establishes compute-optimal scaling laws for both stages.
The problem it solves
High computational cost and inefficiency in training visual generation models that require both high-level semantic understanding and fine-grained pixel details.
What you could use it for
- Training efficient text-to-image models with reduced compute costs
- Developing semantic tokenizers for visual generation without reconstruction losses
- Optimizing the balance between knowledge acquisition and rendering fidelity in generative AI
Why it matters
Significantly reduces the computational cost of training high-quality text-to-image models by identifying optimal scaling laws and demonstrating that semantic quality matters more than AR-only generation quality.
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.