Test-Time Prompt-Agnostic Decomposition
What it is
This paper proposes TPD, a framework to stabilize test-time prompt tuning by analyzing prompt updates as a dynamical system. It decomposes updates into magnitude and direction components, using Adaptive Koopman Control to shrink expansive updates and a Hankel Update Router to filter out oscillatory noise.
The problem it solves
Existing test-time prompt tuning methods suffer from cumulative errors, specifically magnitude expansion and directional drift, which destabilize the prompt update trajectory when using noisy unlabeled objectives.
What you could use it for
- Stabilizing test-time adaptation for vision models facing distribution shift
- Improving the robustness of text-based prompt tuning methods
- Preventing accuracy degradation caused by noisy unlabeled data during test-time updates
Why it matters
Provides a general, plug-and-play mechanism to improve the stability and accuracy of test-time adaptation across various modalities and backbone architectures without requiring changes to the model or loss function.
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.