Perceiving and Predicting Dynamic Sports: Spatiotemporal Intelligence in Vision Models for Egocentric Tennis Video
What it is
This paper investigates how well foundation vision models can interpret and predict dynamic scenes from egocentric video, a critical capability for agile robots. It introduces a synchronized tennis dataset and evaluates seven spatiotemporal prediction tasks, including 3D localization and trajectory forecasting, using image encoders, video encoders, and vision-language models.
The problem it solves
It remains unclear how well current foundation vision models support the spatiotemporal intelligence required to interpret scenes, reason about interactions, and anticipate future events from a moving egocentric camera perspective.
What you could use it for
- Training robots to anticipate ball trajectories in dynamic sports environments
- Evaluating the spatiotemporal reasoning capabilities of vision-language models
- Generating real-time whole-body kinematic motion from egocentric video predictions
Why it matters
Provides a benchmark for understanding how foundation vision models support real-time dynamic robots, potentially enabling more agile and responsive autonomous systems in unstructured environments.
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.