When VLMs Destroy What They Know: Spatial Representations Fall Below Lexical Priors
What it is
This paper introduces EICS, a metric for testing representation stability, and CASP-VL, a benchmark of causally controlled spatial perturbations. It demonstrates that Vision-Language Models (VLMs) fail to faithfully encode spatial relations in their internal representations, even when they perform well on output-based compositional benchmarks.
The problem it solves
Standard output-based evaluations fail to detect that VLMs do not actually preserve relational information in their internal representations, leading to an overestimation of their true reasoning capabilities.
What you could use it for
- Evaluating the internal representational fidelity of Vision-Language Models
- Diagnosing why VLMs fail on compositional spatial reasoning tasks
- Benchmarking the stability of concept encoding under controlled interventions
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.