Feature Information Dynamics in Diffusion
What it is
This paper introduces an information-theoretic framework to quantify when specific features are generated during the diffusion denoising process. It uses the I-MMSE identity to connect feature mutual information changes to the gap between unconditional and feature-conditional denoising losses, providing practical estimators for feature information density.
The problem it solves
The observation that diffusion models generate coarse structure before fine detail is largely empirical and qualitative, lacking a formal theoretical framework to localize and measure feature generation timing.
What you could use it for
- Analyzing the temporal order of feature generation in pixel diffusion models
- Comparing information dynamics across different latent representations like SDVAE, VAVAE, and RAE
- Designing ordered generation strategies to improve diffusion model training
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.