Who YouDub-webui is for#
Content creators publishing multilingual video regularly
Creators who publish to both YouTube and Bilibili can run YouDub-webui on a local machine or small server to automate dubbing across new uploads and back-catalog videos. The author's 20,000+ video Bilibili channel runs entirely on this pipeline.
Skip if:
If you process fewer than a handful of videos per month, a one-off cloud service may be simpler than the multi-step local setup and the GPU infrastructure overhead.
Developers building video localization microservices
The FastAPI backend and modular pipeline design make YouDub-webui a reasonable starting point for a production localization service. Each stage (ASR, translation, TTS, mixing) is clearly separated, and the OpenAI-compatible translation hook makes swapping providers straightforward.
Skip if:
If you need managed infrastructure, SLA guarantees, or a multi-tenant architecture out of the box, a commercial API is a faster starting point than building on YouDub-webui.
Educators and language learners creating dubbed study material
Teachers and learners can use YouDub-webui to create dubbed versions of YouTube or Bilibili content for study use, preserving the original background audio while replacing the speech track in the target language.
Skip if:
If your source material is not on YouTube or Bilibili and you do not want to manage local file uploads, the two-entry-point design limits your workflow options.
The problem it solves#
Localizing a video into another language requires chaining six distinct processes: downloading the source video, separating speech from background audio, transcribing speech with accurate word-level timestamps, translating the transcript sentence by sentence, generating dubbed speech that matches the original speaker's voice, and mixing the result with background audio before burning subtitles. Assembling these tools into a reliable pipeline is significant engineering work.
Commercial platforms like Rask AI, HeyGen, Maestra, and Dubverse handle this as managed cloud services, but charge per-minute or per-seat fees that scale with video volume. For a content creator publishing dozens of localized videos per month, a cloud service's free tier caps quickly and paid plans add up. Data residency is a secondary concern: uploading video content to a third-party cloud means your footage, speaker voice, and transcript all leave your infrastructure.
How it solves it#
End-to-End Localization Pipeline
From YouTube or Bilibili URL to dubbed, subtitle-burned mp4 with no manual steps between. The pipeline runs download, source separation, ASR, translation, TTS, mixing, and subtitle burn as a single task. The web UI shows per-stage progress, timing, and logs; failed tasks resume from the failed stage without rerunning earlier steps.
Whisper Speech Recognition with Word-Level Timestamps
Transcribes speech using openai-whisper, defaulting to the large-v3-turbo model. Output includes word-level timestamps, which are used to cut the original vocal track into per-sentence reference segments for the voice cloning stage. Runs on CUDA or CPU; MPS devices fall back to CPU for the float64-dependent timestamp alignment step.
VoxCPM2 Voice Cloning TTS
Generates dubbed speech in the target language using VoxCPM2, cloning the original speaker's voice from reference audio segments cut from the source track. Dubbed segments are aligned to fit the original sentence durations, then mixed with the preserved background audio and burned into the final video file.
Demucs Vocal and Background Separation
Separates the original video's vocal track from background music and sound effects using a Demucs source submodule. The background audio is preserved through the pipeline and remixed with the dubbed vocals in the final output, so background music and ambient sound carry through to the localized version.
Resumable Task Queue with Stage-Level Logs
Failed tasks restart from the stage that failed, reusing outputs from completed stages cached in the local work folder. The web UI displays per-task stage status, stage duration, and full running logs. Tasks can be deleted individually, including their log records and work folder contents, or rerun from scratch.
Local-First Storage with Configurable Translation API
All intermediate files, the SQLite task database, YouTube cookies, and final output videos stay on your machine. Translation uses any OpenAI-compatible Chat Completions endpoint you configure, including locally run models. The backend enforces Argon2id password authentication and per-session CSRF protection on all API endpoints.
Strengths and trade-offs#
Strengths
- Production-Verified at ScaleThe author's Bilibili channel has published 20,000+ videos, all dubbed using YouDub-webui. That track record confirms the pipeline handles regular, high-volume content production reliably, not just a proof-of-concept demo.
- No Per-Minute Fees or Upload LimitsRask AI and HeyGen charge per minute of video processed. YouDub-webui runs on your own hardware, so your only ongoing costs are GPU compute time and the API fees for your chosen translation model. At high video volumes, the self-hosted path is significantly cheaper than any managed subscription.
- Modular Pipeline with Swappable ComponentsThe ASR, translation, and TTS stages are independently replaceable. You can substitute a different ASR backend for Whisper, swap VoxCPM2 for another TTS system, or point the translation stage at any OpenAI-compatible endpoint including locally-run models. Clear stage boundaries in the FastAPI backend make each substitution targeted.
- Apache-2.0 License Permits Commercial UseApache-2.0 allows commercial use, modification, and distribution without a copyleft requirement. You can run YouDub-webui in a commercial content production operation or build services on top of it without releasing your changes, as long as you retain the original license notice.
Trade-offs
- -GPU Required for Practical UseCPU-only operation is supported but the README describes it as very slow for full transcription, source separation, and TTS workflows. A CUDA-capable NVIDIA GPU is recommended for any real video localization pipeline. Apple Silicon (MPS) is partially supported: Whisper falls back to CPU for word-level timestamp alignment due to MPS float64 constraints.
- -Multi-Step Manual Setup Without DockerThere is no Docker image. Setup requires Python 3.12, Node.js 20+, FFmpeg, a Python virtual environment, submodule initialization for Demucs, environment variable configuration, and an Argon2id password hash generated interactively. The backend and frontend start as separate processes. The process involves significant manual steps across platforms.
- -YouTube Access Requires Proxy and CookiesProcessing YouTube videos requires a locally accessible proxy and Netscape-format YouTube cookies exported from a logged-in browser session. Teams in regions without unrestricted YouTube access, or those without a suitable proxy, cannot process YouTube URL tasks without additional network configuration.
YouDub-webui vs alternatives#
YouDub-webui vs Rask AI
Rask AI is a managed cloud service for AI video localization, supporting dozens of languages through a web-based upload workflow. YouDub-webui addresses the same core need: take a video with speech in one language and produce a dubbed version in another. The key differences are deployment model, data residency, and cost structure.
| Feature | YouDub-webui | Rask AI |
|---|---|---|
| License | Apache-2.0 (open source) | Proprietary |
| Deployment | Self-hosted, local | Cloud-only |
| Pricing | Compute costs only | Per-minute subscription |
| Data residency | Local machine | Rask AI servers |
| GPU required | Yes (recommended) | No |
YouDub-webui is the stronger choice when you are processing a large volume of videos regularly and want to avoid per-minute fees. If you publish dozens of videos per month, the subscription cost of a managed service grows linearly while your compute cost stays roughly fixed. The trade-off is setup effort: YouDub-webui requires a machine with a capable GPU and a multi-step installation.
Rask AI is worth considering if you need a no-infrastructure option, support for many language pairs with no local model setup, or a managed product with customer support.
YouDub-webui vs HeyGen
HeyGen provides video translation and dubbing through its Video Translate feature, alongside its core AI avatar product. Like Rask AI, it is a cloud-only proprietary service with credit or subscription billing. The core difference from YouDub-webui is that HeyGen processes your video content on its own servers.
| Feature | YouDub-webui | HeyGen |
|---|---|---|
| License | Apache-2.0 | Proprietary |
| Deployment | Self-hosted | Cloud-only |
| Data residency | Local machine | HeyGen's servers |
| Input sources | YouTube URL, Bilibili URL, local upload | Upload only |
| Pricing | Compute costs only | Credit-based |
YouDub-webui is the better fit for creators who need to keep video content on their own infrastructure, or who process a volume that makes per-credit billing prohibitive. HeyGen is the better choice for teams who want a polished managed product with no local GPU infrastructure to maintain.
Install and self-host#
Self-hosting requires cloning the repository with its Demucs submodule, then installing Python and Node.js dependencies before starting the backend and frontend services.
```bash
git clone https://github.com/liuzhao1225/YouDub-webui.git
cd YouDub-webui
git submodule update --init --recursive
python3.12 -m venv .venv
.venv/bin/pip install -r requirements.txt
npm --prefix apps/web install
cp env.txt.example .env
```What it's built on#
- Languages
- PythonTypeScript
- Frameworks
- FastAPINext.jsReact
- Tooling
- FFmpeg
FAQ#
Does YouDub-webui support languages other than English and Chinese?
The README documents two primary language routes: YouTube English-to-Chinese and Bilibili Chinese-to-English. Translation uses any OpenAI-compatible Chat Completions endpoint, so other language pairs work if your chosen model supports them. Whisper handles many languages for ASR. VoxCPM2 TTS quality for languages beyond the primary two has not been validated in the documented production case, so additional testing is advised before relying on other language pairs at scale.
Is a GPU required to run YouDub-webui?
A CUDA-capable NVIDIA GPU is strongly recommended. The README states that running Whisper transcription, Demucs source separation, and VoxCPM2 TTS on CPU is very slow for real video workflows. Setting DEVICE=cpu is supported for partial testing, but a complete video localization pipeline on CPU takes significantly longer per video. Apple Silicon (MPS) is partially supported: VoxCPM2 uses MPS, but Whisper runs on CPU on MPS devices due to float64 constraints in the word-level timestamp alignment step.
What does the Apache-2.0 license mean for commercial use?
Apache-2.0 permits commercial use, modification, and distribution without a copyleft requirement. You can run YouDub-webui in a commercial content production operation or build a service on top of it without releasing your own changes as open source, as long as you retain the original Apache-2.0 license notice in any distribution.
Can I upload my own video files instead of using a YouTube or Bilibili URL?
Yes. YouDub-webui accepts local video uploads up to 4 GiB by default (configurable via LOCAL_UPLOAD_MAX_BYTES). You can also upload a pre-translated SRT subtitle file alongside a local video to skip the Whisper transcription and OpenAI translation stages, running only the voice cloning and subtitle burning steps. Uploading an SRT alongside a URL-based task is not supported in the current version.
How does YouDub-webui handle a task that fails partway through?
Failed tasks can be resumed from the stage that failed without re-running earlier stages. The web UI displays per-stage status, duration, and full logs. Intermediate outputs from completed stages are cached in the local work folder and reused on resume. You can also delete a task's records and output files from the UI if you want to start fresh.
Similar open-source tools#
stt
Local speech-to-text with multi-format output and GPU acceleration
pyvideotrans
Open source video translation with AI dubbing and voice cloning
dograh
Open source voice agent builder with telephony and BYOK AI support
FunClip
Clip video by text or speaker with local AI transcription
whishper
Local speech-to-text transcription and subtitling with a web UI
willow
Self-hosted Amazon Echo alternative for $50 ESP32 hardware