HAD-VLM: Hybrid Autoregressive-Diffusion Framework for Medical Vision-Language Models
What it is
This paper introduces HAD-VLM, a unified vision-language model that combines autoregressive text generation with a diffusion process to predict continuous numerical measurements from medical images. It addresses the mismatch between discrete token-based language models and continuous measurement targets by using a shared backbone with visual prompt tuning and a coordinate-length consistency mechanism.
The problem it solves
Standard discrete token-based vision-language models struggle to accurately predict continuous numerical measurements (like anatomical dimensions) due to the inherent mismatch between discrete text tokens and continuous physical quantities.
What you could use it for
- Quantitative echocardiographic measurement (e.g., IVSd, LVIDd, LVIDs, LVPWd)
- X-ray vertebral measurement
- Medical question answering with quantitative grounding
Why it matters
Significantly reduces measurement error in quantitative medical imaging (21.4-37.6% reduction vs. autoregressive baselines) while maintaining the flexibility of prompt-conditioned task execution, potentially improving diagnostic accuracy and workflow efficiency in clinical settings.
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.