Recursive Video In-Context Learning for Agentic Robot
What it is
This paper introduces RV-ICL, a training-free method that structures demonstration videos into a hierarchical format of keyframes, phases, and clips. It allows LLM agents to navigate this hierarchy via read-only tools, loading only the specific video details needed for the current sub-goal rather than processing the full video.
The problem it solves
Full demonstration videos are too large for LLM context windows, while fixed keyframes lose critical contact details needed for precise manipulation, and static prompts fail to adapt to the shifting information needs during execution.
What you could use it for
- Improving the success rate of robot manipulation tasks using single demonstration videos
- Enabling LLM agents to access fine-grained visual context during execution without bloating the prompt
- Enhancing the performance of frozen vision-language-action policies in long-horizon tasks
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.