AVPS-360: A Spatial Audio-Visual Benchmark for Instance Segmentation and Tracking of Sound-Producing Persons
What it is
AVPS-360 is a spatial audio-visual benchmark dataset that pairs first-order Ambisonics audio with synchronized 360-degree video to evaluate audio-visual instance segmentation (AVIS). It addresses the limitations of monaural audio benchmarks by providing inter-channel cues for distinguishing concurrent sound sources in real-world interactions.
The problem it solves
Existing audio-visual instance segmentation benchmarks predominantly use monaural audio, which lacks the inter-channel spatial cues necessary to distinguish concurrent sound sources in complex real-world scenes.
What you could use it for
- Evaluating audio-visual instance segmentation models on spatial audio data
- Benchmarking sound event localization and detection (SELD) methods
- Training models to track sound-producing persons in 360-degree video environments
Why it matters
Enables the development of more robust audio-visual AI systems that can accurately localize and track sound sources in immersive 360-degree environments, bridging the gap between AVIS and SELD research.
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.