Rephrase Before You Act: Characterizing and Mitigating Language Sensitivity in Vision-Language-Action Models
What it is
This paper identifies that Vision-Language-Action (VLA) models are highly sensitive to minor changes in instruction phrasing, which can drastically alter task success rates. It proposes a mitigation strategy that uses a large language model to distill rephrasing rules from training data, which are then applied to incoming instructions at deployment time to improve robustness without retraining the policy.
The problem it solves
Vision-language-action models exhibit high sensitivity to instruction phrasing, where minor semantic edits can cause large drops in task success rates, limiting their robustness and generalization.
What you could use it for
- Improving the reliability of robotic manipulation models like π0 and π0.5
- Reducing the performance gap between in-distribution and out-of-distribution tasks in VLA models
- Enhancing the robustness of vision-language-action systems against adversarial or varied natural language instructions
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.