SonoWorld4D: From One Video to a 4D Audio-Visual Scene via Diffusion Synchronization
What it is
This paper introduces SonoWorld4D, a framework that converts a single monocular video into an explorable 4D audio-visual scene. It works by reconstructing motion trajectories for dynamic objects and synthesizing a physically grounded sound field using dual-space audio diffusion synchronization.
The problem it solves
Prior 4D scene generation methods largely ignored sound, failing to model the intrinsic coupling between dynamic visual events and their corresponding audio sources.
What you could use it for
- Generating immersive 360-degree audio-visual environments from standard video footage
- Creating realistic spatial audio for virtual reality or augmented reality applications
- Reconstructing dynamic scenes with synchronized sound and visual motion for simulation
Why it matters
Enables the creation of fully immersive, physically consistent 4D worlds from minimal input, bridging the gap between visual generation and spatial audio synthesis.
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.