Denoising Hierarchical Representations: Joint Continuous Diffusion for Language Modeling
What it is
This paper introduces Hierarchical Continuous Diffusion Language Models (H-CDLMs), a framework that improves continuous diffusion language models by jointly diffusing tokens and coarser semantic clusters in parallel. It demonstrates that this hierarchical approach yields significant performance gains in generative perplexity and reasoning tasks compared to baseline continuous models.
The problem it solves
Continuous diffusion language models often underperform discrete models in terms of generative quality and reasoning accuracy due to the difficulty of modeling complex token dependencies in a continuous space.
What you could use it for
- Improving the quality of text generation in continuous diffusion language models
- Enhancing reasoning capabilities in flow-matching language models
- Developing order-agnostic parallel text generation systems
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.