LeWAM: A JEPA World Action Model with Diffusion-Steering-Based MPC
What it is
LeWAM is a bidirectional transformer that unifies forward, backward, inverse dynamics, and policy prediction within a single decoder-free JEPA latent space. It uses diffusion-steering-based Model Predictive Control (MPC) to plan in the noise space of the policy head, which improves closed-loop performance compared to sampling raw actions.
The problem it solves
Reconstruction-based world action models carry noisy and redundant information that complicates downstream predictions, and standard MPC planning in raw action space can exploit dynamics-model inaccuracies, leading to poor closed-loop performance.
What you could use it for
- Robotic manipulation tasks requiring both action execution and future state prediction
- Planning in complex environments where reconstruction-based world models suffer from noisy representations
- Improving the robustness of Model Predictive Control in robot policies by planning in latent noise space
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.