VideoHarness-RSI: Recursive Harness Self-Improvement for Long-Video Understanding
What it is
This paper introduces VideoHarness-RSI, a method for automatically optimizing the context-construction logic (harness) used to feed long videos into frozen vision-language models. It uses an outer-loop proposer to recursively search for executable programs that better select and structure video evidence, improving performance without updating the underlying model weights.
The problem it solves
Existing long-video understanding methods often couple context construction with model training or manual design, making it difficult to isolate and optimize the specific logic for selecting relevant video segments within limited context windows.
What you could use it for
- Improving long-video understanding benchmarks without retraining VLMs
- Automating the design of inference pipelines for video agents
- Transferring optimized context strategies across different video datasets
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.