UltraEval-Audio
What it is
UltraEval-Audio is a unified open-source framework for evaluating audio foundation models, specifically designed for large audio models covering ASR, TTS, and Audio Codecs. It aggregates 34 authoritative benchmarks and provides an isolated runtime mechanism to manage model-specific dependencies and prevent conflicts during evaluation.
The problem it solves
Lack of a unified, reproducible framework to evaluate diverse audio models (ASR, TTS, Codec) across multiple benchmarks without manual dependency management and data processing.
What you could use it for
- Benchmarking open-source ASR models on multilingual datasets
- Evaluating TTS systems for zero-shot voice cloning and multilingual synthesis
- Assessing audio codec performance and audio LLM capabilities
- Reproducing evaluation results for popular models like CosyVoice and Qwen3-ASR
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.