MemVLN: Streaming Episodic Memory for Vision-Language Navigation
What it is
MemVLN is a framework for vision-language navigation that introduces a streaming episodic memory module to help agents maintain context over long-horizon tasks. It works by compressing perceptual features into bounded memory tokens using cross-attention and training via a masking-based trajectory summarization task.
The problem it solves
VLM-based streaming agents struggle to maintain persistent episodic memory during long-horizon navigation tasks, leading to poor performance in complex 3D environments.
What you could use it for
- Enabling embodied agents to navigate 3D environments using language instructions
- Improving success rates in long-horizon vision-language navigation benchmarks like VLN-CE
- Post-training VLMs for physical robot navigation in real-world scenes
Why it matters
Significantly improves the success rate of vision-language navigation agents in both simulated and physical environments by providing a mechanism for persistent, bounded episodic memory.
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.