VibeVoice

Open source alternative to ElevenLabs, Microsoft Azure AI Speech and PlayHT

55.9K stars
MIT
Active this month
Last commit 29 days ago

Last verified Nov 6, 2026

Generate long-form, multi-speaker text-to-speech and 60-minute speech recognition audio with Microsoft's MIT-licensed frontier voice AI family.

Visit website GitHub55.9K

Read the full breakdownWho it is for, how it works, trade-offs, the quick start and the FAQ

who it's for

Who VibeVoice is for#

ML engineers building podcast generation tools

VibeVoice-TTS supports up to 4 speakers in a single 90-minute generation pass, maintaining turn-taking and conversational dynamics without manual audio stitching. Teams building automated podcast creation pipelines can feed a transcript and receive a coherent multi-voice audio file without managing segment boundaries.

Skip if:

If your production environment requires commercial SLAs, guaranteed uptime, or voice cloning from a custom speaker sample, ElevenLabs or PlayHT are more mature choices for that specific need.

Research teams running meeting transcription pipelines

VibeVoice-ASR handles 60-minute recordings in a single inference pass, labeling each speaker segment with who spoke, when, and what they said. The customized hotwords feature improves accuracy on technical vocabulary, project names, or internal terms that generic ASR models handle poorly.

Skip if:

If your recordings exceed 60 minutes, you will need to segment them before passing to the model. For speaker enrollment from a named reference audio, the current ASR model does not support that path.

Developers prototyping voice interfaces on CPU hardware

The VibeVoice-ASR-BitNet variant compresses the ASR model to 1.58 GB and runs at real-time speed on 3 or more CPU threads, removing the GPU dependency. This makes it practical for edge devices, local developer environments, or embedded scenarios where GPU resources are not available.

Skip if:

For production deployment requiring streaming voice commands at low latency alongside other CPU-intensive work, the CPU inference may not leave enough headroom. Measure RTF under your target load before committing.

the problem

The problem it solves#

Most text-to-speech APIs and speech recognition services impose hard limits that break long-form audio workflows. Commercial TTS platforms cap output at a few minutes per request, forcing developers to stitch together audio segments that lose speaker consistency and conversational rhythm across the seams. Multi-speaker generation typically supports only 1-2 speakers, which rules out realistic podcast or panel conversation audio.

On the transcription side, most ASR systems slice long recordings into 30-second chunks before processing them, discarding the global context that makes accurate diarization possible. You get a transcript, but not reliably a record of who said what with consistent speaker labels across an hour-long meeting. Adding accurate timestamps, handling domain-specific terms, and streaming results as the audio arrives are separate problems each requiring a different tool or service integration.

how VibeVoice solves it

How it solves it#

Long-Form Multi-Speaker TTS

Synthesizes conversational audio up to 90 minutes in a single pass with up to 4 distinct speakers. The next-token diffusion framework uses a large language model to maintain textual and dialogue context, so speaker turns and emotional dynamics remain consistent across the full generation rather than drifting after a few minutes.

60-Minute Single-Pass ASR with Diarization

Processes up to 60 minutes of continuous audio within 64K tokens, producing structured output with speaker labels (Who), timestamps (When), and transcript text (What). Unlike chunk-based ASR that loses cross-segment speaker tracking, VibeVoice-ASR maintains diarization coherence across the full recording.

Real-Time Streaming TTS at ~300ms Latency

VibeVoice-Realtime-0.5B accepts streaming text input and begins generating audible speech in approximately 300 milliseconds, handling up to 10 minutes of continuous output. At 0.5B parameters it is deployment-friendly compared to the full 1.5B TTS model, and runs on Colab or modest hardware.

CPU-Only Edge ASR via BitNet Quantization

VibeVoice-ASR-BitNet compresses the 7B model from 4.62 GB to 1.58 GB using heterogeneous quantization (I8_S + I2_S), achieving real-time inference with RTF under 1 on 3+ CPU threads. This eliminates the GPU requirement for ASR inference, making on-device or edge deployment practical without specialized hardware.

Multilingual ASR for 50+ Languages

VibeVoice-ASR handles over 50 languages natively, with customizable hotwords for domain-specific vocabulary, technical terms, or proper nouns. The streaming variant (VibeVoice-ASR-Streaming) continuously emits text chunks as audio arrives, with explicit speaker labeling across 10 supported languages.

strengths · trade-offs

Strengths and trade-offs#

Strengths

MIT License with No API FeesThe entire model family ships under MIT, so you can run inference on your own hardware without per-character, per-minute, or seat-based billing. Paid alternatives like ElevenLabs charge by character and PlayHT by minute; VibeVoice's only cost is the compute you provision.
Long-Form Limits That Exceed Commercial APIsThe 90-minute TTS and 60-minute ASR limits significantly exceed what most commercial APIs allow in a single request. For podcast generation, long-form meeting transcription, or audiobook production, this removes the stitching logic that most developers build around shorter API caps.
Azure AI Foundry Labs Integration for Managed AccessVibeVoice-ASR is integrated into Azure AI Foundry Labs as of March 2026, so teams that need managed access or want to test the model without self-hosting infrastructure can use it through Microsoft's platform while the same weights remain self-hostable from Hugging Face.
Active Research Trajectory with Multiple Model VariantsSince its August 2025 release, the family has expanded to include streaming ASR, a BitNet CPU variant, a realtime TTS model, and Hugging Face Transformers integration. The ASR paper is on arXiv and the TTS paper earned an Oral acceptance at ICLR 2026.

Trade-offs

-TTS Code Removed from Repo; Research-Only RecommendationThe VibeVoice-TTS source code was removed from the GitHub repository in September 2025 after the team identified out-of-scope usage. The TTS model weights remain available on Hugging Face, but the repo no longer includes TTS training or inference code, and the Quick Try entry in the README is listed as Disabled. Microsoft explicitly states the models are intended for research and development only and does not recommend production deployment without additional testing.
-GPU Required for Full-Scale InferenceThe 7B ASR model and the 1.5B TTS model require significant GPU memory for full-speed inference. The BitNet CPU variant reduces this for ASR, but TTS at production scale still demands a GPU. Teams running inference on shared cloud compute should budget for the memory footprint before integrating into a service.
-132 Open IssuesThe repository had 132 open issues at the time of indexing. As a relatively recent Microsoft research release with rapid expansion into new model variants, documentation gaps and integration rough edges are expected.

versus alternatives

VibeVoice vs alternatives#

VibeVoice vs ElevenLabs

Both generate expressive TTS, but ElevenLabs is a managed API with voice cloning and production SLAs; VibeVoice is a self-hostable research model.

Feature	VibeVoice	ElevenLabs
License	MIT	Proprietary
Self-hosting	Yes	No
Max TTS duration	90 minutes	Per-request API limit
Pricing	Compute cost	Per-character billing
Voice cloning	Not supported	Yes

VibeVoice suits teams needing long-form multi-speaker generation on their own infrastructure without per-character costs. ElevenLabs is better when you need voice cloning, a broad managed voice library, or a production API with guaranteed uptime.

VibeVoice vs Azure AI Speech

Azure AI Speech is Microsoft's commercial TTS and STT service. VibeVoice is Microsoft's MIT-licensed research counterpart, with self-hostable weights on Hugging Face.

Feature	VibeVoice	Azure AI Speech
License	MIT	Proprietary
Self-hosting	Yes	No
ASR single-pass limit	60 minutes	Chunked processing
Production SLA	None	Yes
Hotword customization	Yes	Yes

VibeVoice-ASR is also available through Azure AI Foundry Labs for teams that prefer managed access to the same underlying model. Azure AI Speech is the right choice for production applications requiring certified SLAs, enterprise support, or compliance documentation.

VibeVoice vs PlayHT

PlayHT serves high-volume content production with voice cloning and 140+ language TTS. VibeVoice targets research workloads with long-form generation.

Feature	VibeVoice	PlayHT
License	MIT	Proprietary
Self-hosting	Yes	No
Max TTS duration	90 minutes	Per-request API limit
Multilingual TTS	English, Chinese, others	140+ languages
Voice cloning	Not supported	Yes
Pricing	Compute cost	Per-minute billing

PlayHT is better for teams needing a managed API for audio content at scale, voice cloning, or broad multilingual coverage. VibeVoice fits research and developer workloads where self-hosting and long-form generation matter more than API convenience.

install · quick start

Quick start#

bash

Install VibeVoice-ASR via the Hugging Face Transformers library for speech recognition, or clone the repository to access realtime TTS notebooks and documentation.

```bash
pip install transformers
git clone https://github.com/microsoft/VibeVoice
```

tech stack · detected from GitHub

What it's built on#

Languages: Python
Frameworks: FastAPI

frequently asked

FAQ#

Is VibeVoice free to use?

Yes. VibeVoice is MIT licensed, so you can run the models on your own infrastructure without licensing fees. The model weights are available on Hugging Face at no cost. Microsoft notes that the models are intended for research and development purposes and does not recommend production deployment without additional testing.

Why was the VibeVoice TTS code removed from GitHub?

In September 2025, Microsoft discovered the tool was being used in ways inconsistent with its stated intent as a research framework for the speech synthesis community. The TTS source code was removed from the repository while Microsoft assessed controls to prevent out-of-scope use. The model weights remain available on Hugging Face, but the Quick Try for VibeVoice-TTS-1.5B in the README is currently listed as Disabled.

Can VibeVoice-ASR run without a GPU?

Yes, via the VibeVoice-ASR-BitNet variant. It uses heterogeneous quantization (I8_S + I2_S) to compress the model from 4.62 GB to 1.58 GB, running at real-time speed (RTF under 1) on 3 or more CPU threads. Standard VibeVoice-ASR-7B benefits from GPU for practical inference speed.

What languages does VibeVoice support?

VibeVoice-ASR supports over 50 languages for speech recognition. The streaming ASR variant (VibeVoice-ASR-Streaming) supports 10 languages for continuous transcription. The VibeVoice-Realtime TTS model added experimental multilingual voices in nine languages (German, French, Italian, Japanese, Korean, Dutch, Polish, Portuguese, Spanish) and 11 English style voices as of December 2025.

How does VibeVoice-ASR handle speaker identification in long recordings?

VibeVoice-ASR jointly performs speech recognition, diarization, and timestamping in a single pass, outputting structured results for who spoke (speaker label), when they spoke (timestamps), and what they said (transcript). You can supply customized hotwords before inference to improve accuracy on domain-specific vocabulary, proper nouns, or technical terms. The model accepts up to 60 minutes of continuous audio within 64K tokens.

also worth a look

Similar open-source tools#

Fish Speech

Alternative toElevenLabs

Voice-cloning TTS with emotion tags and multi-character scenes

33,590 GitHub stars
Licence Fish Audio Community License v1.0

F5-TTS

Alternative toElevenLabs

Open source TTS with zero-shot voice cloning and multilingual support.

15,513 GitHub stars
Licence MIT

VoiceStudio

Alternative toElevenLabs

Voice cloning, dubbing, and transcription that runs fully offline

101,036 GitHub stars
Licence AGPL-3.0

willow

Alternative toGoogle Home

Self-hosted Amazon Echo alternative for $50 ESP32 hardware

3,131 GitHub stars
Licence Apache-2.0

supertonic

Alternative toElevenLabs

Instantly convert text to speech with customizable options

13,772 GitHub stars
Licence MIT

VoxCPM

Alternative toElevenLabs

Tokenizer-free multilingual text-to-speech with voice cloning

41,025 GitHub stars
Licence Apache-2.0

by category

Open source tools in these categories#

503

AI & Machine Learning

freeCodeCamp, REA (Reverse Engineer Anything), OpenClaw

Inbox Zero

Alternative toSuperhuman

AI email assistant that drafts replies and clears your inbox

12,915 GitHub stars
Licence AGPL-3.0
#Email

rakazo

Alternative toGrok Bot

AI teammates you own: your keys, your model, your machine.

7,028 GitHub stars
Licence Apache-2.0
#AI Agents

SnitchFeed

Live monitoring on LinkedIn, X, Reddit & more. Agentic social listening for startups.

Marketing & Growth

Advertise your product

The projects we add and every new guide. Free, unsubscribe any time.

Find open source alternatives to paid software

VibeVoice vs ElevenLabs

Both generate expressive TTS, but ElevenLabs is a managed API with voice cloning and production SLAs; VibeVoice is a self-hostable research model.

Feature	VibeVoice	ElevenLabs
License	MIT	Proprietary
Self-hosting	Yes	No
Max TTS duration	90 minutes	Per-request API limit
Pricing	Compute cost	Per-character billing
Voice cloning	Not supported	Yes

VibeVoice vs Azure AI Speech

Azure AI Speech is Microsoft's commercial TTS and STT service. VibeVoice is Microsoft's MIT-licensed research counterpart, with self-hostable weights on Hugging Face.

Feature	VibeVoice	Azure AI Speech
License	MIT	Proprietary
Self-hosting	Yes	No
ASR single-pass limit	60 minutes	Chunked processing
Production SLA	None	Yes
Hotword customization	Yes	Yes

VibeVoice vs PlayHT

PlayHT serves high-volume content production with voice cloning and 140+ language TTS. VibeVoice targets research workloads with long-form generation.

Feature	VibeVoice	PlayHT
License	MIT	Proprietary
Self-hosting	Yes	No
Max TTS duration	90 minutes	Per-request API limit
Multilingual TTS	English, Chinese, others	140+ languages
Voice cloning	Not supported	Yes
Pricing	Compute cost	Per-minute billing