VibeVoice
What it is
VibeVoice is a family of open-source voice AI models for both Text-to-Speech (TTS) and Automatic Speech Recognition (ASR). It uses continuous speech tokenizers at a low frame rate (7.5 Hz) and a next-token diffusion framework to handle long-form audio and high-fidelity speech synthesis.
The problem it solves
Addresses the computational inefficiency and fidelity loss in processing long-form audio by using ultra-low frame rate tokenizers and a unified diffusion-based approach.
What you could use it for
- Long-form speech synthesis (up to 90 minutes) with multiple distinct speakers.
- Unified speech-to-text transcription for long-form audio (up to 60 minutes) with speaker and timestamp labeling.
- Real-time text-to-speech with streaming input support.
- Edge CPU inference for speech recognition using quantized models.
Read the source ↗Back to the library →
Summary written by an AI model from the source; check the source before you rely on it.