MoETok: Sparse Expert Routing for 1D Visual Tokenizers
What it is
MoETok is a 1D visual tokenizer that uses Mixture-of-Experts (MoE) routing to process image tokens based on their learned roles. It distinguishes between 'Content' queries for high-level semantics and 'Detail' queries for fine-grained reconstruction, routing them through specialized experts to improve both representation and generation quality.
The problem it solves
Standard 1D visual tokenizers treat all learnable queries uniformly, failing to adapt processing to the distinct roles (semantic vs. detail) that queries spontaneously develop, which limits their performance in both reconstruction and downstream generation tasks.
What you could use it for
- Improving the quality of discrete visual tokenizers for autoregressive image generation models
- Enhancing the semantic representation of visual tokens for unified understanding and generation models
- Optimizing the trade-off between reconstruction fidelity and semantic abstraction in 1D visual encoders
Why it matters
If successful, MoETok could enable discrete autoregressive models to match or surpass diffusion-based models in image generation quality while maintaining efficient, sparse computation, potentially unifying visual understanding and generation in a single architecture.
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.