Agentic RSR: Real-to-Sim-to-Real through Scene Reconstruction and Execution-Grounded Robot Policies
What it is
This paper presents Agentic RSR, a framework that unifies scene reconstruction, policy development, and real-robot execution for manipulation tasks. It uses agents to iteratively refine 3D scene reconstructions from video and develop executable policies in simulation, which are then transferred to physical robots with execution-grounded feedback.
The problem it solves
Scene reconstruction and policy development are typically treated as separate processes, and policies developed in simulation often fail to operate effectively on real robots due to discrepancies in observations and task-relevant interactions.
What you could use it for
- Automating the creation of simulation environments from real-world robot workspace videos
- Developing and validating robot manipulation policies in simulation before real-world deployment
- Bridging the sim-to-real gap by using execution feedback to refine policies on physical hardware
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.