PORTAL: A Controller for Test-Time Intervention
What it is
PORTAL is a decision-making framework that determines when to apply test-time interventions, such as updating normalization statistics, to deployed models. It uses active learning to select which inputs to label, allowing it to evaluate multiple candidate prediction rules and stop querying once the optimal choice is clear or the potential gain is insufficient.
The problem it solves
Test-time adaptation methods can improve performance on some batches but degrade it on others, and average accuracy metrics fail to identify when an intervention is harmful or unnecessary.
What you could use it for
- Deploying models in environments with shifting data distributions where blind adaptation might cause harm
- Managing test-time adaptation for corrupted or out-of-distribution image inputs
- Optimizing the cost of human-in-the-loop labeling for model adaptation
Why it matters
It prevents severe performance drops caused by indiscriminate test-time adaptation and maximizes the utility of scarce label feedback by ensuring interventions are only applied when they are beneficial.
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.