Who RealtimeSTT is for#
Developers building voice assistants with wake word activation
RealtimeSTT's wake word detection and low-latency VAD segmentation make it a natural fit for always-listening voice assistant prototypes. You listen continuously, activate on a trigger phrase, and surface realtime partial transcripts to the UI while the speaker is still talking.
Skip if:
If your assistant needs to run on mobile or embedded hardware without a Python runtime, RealtimeSTT does not port to those targets.
Developers building dictation tools or live caption applications
The automatic recording loop and callback API transcribe continuous speech without requiring you to manage audio chunks or threading. Feed results to a text editor, subtitle renderer, or notes app as each utterance completes.
Skip if:
If you need transcription in a language with limited Whisper model support, check accuracy benchmarks for your target language before committing to this pipeline.
Backend engineers integrating STT into existing Python services
External audio input lets you feed audio from a websocket, file, or inter-process pipe rather than a microphone, so RealtimeSTT fits inside existing audio ingestion pipelines without requiring hardware access on the server.
Skip if:
Teams building in Node.js, Go, or other non-Python stacks will need to wrap RealtimeSTT in a sidecar service or use a language-native alternative.
Teams prototyping voice-first browser applications
The FastAPI reference server provides a browser-compatible WebSocket streaming endpoint with multi-user session isolation and shared inference resources. You can demo a voice-driven web interface without building the audio server from scratch.
Skip if:
The FastAPI server is source-only and documented as a reference example, not a production-ready deployment. Production use requires additional hardening not covered by the library's documentation.
The problem it solves#
Adding real-time speech transcription to an application almost always means signing up for a paid cloud API. Services like Deepgram, AssemblyAI, and Google Cloud Speech-to-Text charge by the minute, require an internet connection for every audio request, and send your users' audio through a third-party server. For privacy-sensitive applications, that data flow is a problem. For high-volume workloads, the per-minute billing becomes expensive fast.
Building a self-hosted alternative used to mean wrestling with raw Whisper models, writing your own voice activity detection to split audio into utterances, and managing threading so transcription does not block the main loop. Most teams either paid for the managed API or shipped fragile audio pipelines they never fully trusted.
How it solves it#
Voice Activity Detection with WebRTC and Silero VAD
Automatically segments speech from silence using WebRTC VAD and Silero VAD before passing audio to the transcription engine. This means the model only processes real utterances, keeping latency low and avoiding wasted compute on background noise or pauses between sentences.
Multi-Engine Transcription Backend
faster_whisper is the default backend, but you can switch to whisper.cpp, Moonshine, sherpa-onnx, Kroko-ONNX, Cohere Transcribe, FunASR, or Hugging Face Transformers models via install extras. Match the engine to your accuracy, speed, and licensing requirements without changing application code.
Realtime Partial Transcript Callbacks
Emits partial transcript updates as speech is being spoken, not just a final result when the utterance ends. Your application receives rolling text while the speaker is still talking, which is essential for low-latency captioning and voice UI feedback loops.
Wake Word Activation
Optional wake word detection via Porcupine or OpenWakeWord triggers transcription only when a specific phrase is spoken. This enables always-listening assistants that do not run the transcription engine continuously, reducing CPU and model load when idle.
External Audio Input Without a Microphone
Set use_microphone=False to feed audio from a file, websocket, or another process as 16-bit mono PCM chunks at 16 kHz. RealtimeSTT resamples if needed, so the same API works for live microphone capture and offline or streaming audio pipelines alike.
FastAPI Browser Streaming Server
A reference FastAPI server ships with the repository for browser-based deployments. It includes multi-user session isolation, shared inference resources, health endpoints, and WebSocket streaming, so you can stand up a multi-user voice interface without building the audio backend from scratch.
Strengths and trade-offs#
Strengths
- No Per-Minute API FeesRealtimeSTT processes audio locally, so there are no per-minute charges. Self-hosted deployments run on your own hardware at a fixed infrastructure cost, which is economical compared to Deepgram or AssemblyAI billing at scale. A server handling continuous transcription costs the same whether it processes 10 hours or 10,000 hours in a month.
- Audio Never Leaves Your ServerUnlike cloud STT APIs that route audio through third-party infrastructure, RealtimeSTT never sends audio outside your environment. This is the correct architecture for applications handling sensitive conversations, medical dictation, legal recordings, or deployments subject to data-residency requirements.
- Minimal Code to First TranscriptThe AudioToTextRecorder API wraps all threading, VAD segmentation, and engine management behind a single context manager. A working microphone transcription loop is five lines of Python. The same API handles continuous dictation, realtime partial updates, external audio input, and wake word activation through constructor parameters.
- Active Development with Broad Engine SupportThe project has over 10,000 GitHub stars and last received a commit in June 2026. It supports eleven transcription backends, giving you a migration path if a specific model changes its license, performance characteristics, or availability.
Trade-offs
- -You Manage the Server and Python EnvironmentRunning RealtimeSTT means owning the infrastructure: the server, Python 3.11 environment, PortAudio headers, and optional CUDA drivers. Cloud STT APIs from Deepgram or AssemblyAI handle all of that for you and offer formal SLAs. For teams without DevOps capacity or server access, a managed API is simpler.
- -Python 3.11+ and Platform-Specific System PrerequisitesPython 3.11 or newer is required. On Linux, you must install python3-dev and portaudio19-dev at the system level before pip install. On macOS, Homebrew is needed for PortAudio. CUDA and Windows compatibility require additional setup documented in the installation guide.
- -147 Open Issues and Community-Only SupportWith 147 open GitHub issues, you may encounter edge cases that require a workaround or community investigation. Commercial APIs provide formal support channels, bug SLAs, and dedicated customer success that a community-maintained library cannot match.
RealtimeSTT vs alternatives#
RealtimeSTT vs Deepgram
Deepgram is a managed cloud speech recognition API built for production streaming workloads. Both tools support realtime transcription with low latency, but they operate on opposite models: Deepgram handles infrastructure and charges per minute of audio; RealtimeSTT runs on your own server with no usage fees.
| Feature | RealtimeSTT | Deepgram |
|---|---|---|
| License | MIT | Proprietary |
| Self-hosting | Yes | No |
| Pricing | Free (infrastructure cost) | Per-minute billing |
| Audio data | Stays on your server | Sent to Deepgram |
| Wake word support | Yes (Porcupine, OpenWakeWord) | No native wake word |
RealtimeSTT is the better choice when audio privacy, data sovereignty, or cost control at high volume are the deciding factors. Processing everything locally means audio never leaves your server, which matters for medical, legal, or compliance-sensitive applications.
Deepgram is the better choice when you need managed reliability, formal SLAs, or you are building a production service without the capacity to manage a Python server and model stack. Deepgram also covers a wider range of languages and accents without manual model selection.
RealtimeSTT vs AssemblyAI
AssemblyAI is a cloud STT API focused on transcription accuracy and additional processing features including speaker diarization, sentiment analysis, and entity detection. RealtimeSTT focuses on low-latency streaming transcription without those higher-level features built in.
| Feature | RealtimeSTT | AssemblyAI |
|---|---|---|
| License | MIT | Proprietary |
| Self-hosting | Yes | No |
| Pricing | Free (self-hosted) | Per-minute billing |
| Audio goes to cloud | No | Yes |
| Speaker diarization | No | Yes |
| Post-processing features | No | Yes (sentiment, entities) |
RealtimeSTT is the right pick when you need a streaming transcription loop in Python with no cloud dependency and no per-minute cost. Its multi-engine support means you can swap between faster_whisper and other backends without changing application code.
AssemblyAI is the better fit when you need speaker diarization, entity detection, or post-processing features that would require significant custom work to build on top of raw Whisper output. If you are building a meeting notes or call analysis tool, AssemblyAI's managed feature set may save substantial development time.
Install and self-host#
Install RealtimeSTT via pip with the faster-whisper engine on Python 3.11 or newer.
```bash
pip install "RealtimeSTT[faster-whisper]"
```What it's built on#
- Languages
- Python
- Frameworks
- FastAPI
FAQ#
Is RealtimeSTT free to use?
Yes. RealtimeSTT is MIT licensed, so the library is free to use, modify, and distribute, including for commercial applications. The only cost is the infrastructure you run it on. Some optional engine integrations, such as Porcupine for wake word detection or commercial Kroko models, have their own licensing terms you should review separately.
What transcription engine does RealtimeSTT use by default?
The default engine is faster_whisper, a CTranslate2-optimized version of OpenAI Whisper. Install it with pip install "RealtimeSTT[faster-whisper]". Additional engines including whisper.cpp, Moonshine, sherpa-onnx, Kroko-ONNX, and several Hugging Face Transformers models are available as optional install extras documented in the repository.
Does RealtimeSTT send audio to the cloud?
No. All transcription runs locally on the machine where the library is installed. Audio is never sent to an external server unless you explicitly wire it to one. This makes RealtimeSTT suitable for applications with audio privacy requirements or data-residency constraints that prohibit third-party cloud processing.
How does RealtimeSTT compare to Deepgram or AssemblyAI?
Deepgram and AssemblyAI are managed cloud APIs that charge per minute of audio processed and handle all infrastructure on your behalf. RealtimeSTT is a self-hosted library you run on your own server, with no per-minute fees and no audio leaving your infrastructure. The tradeoff is that you manage the server, Python environment, and model updates yourself. Cloud APIs are the better choice when you need managed reliability, formal SLAs, or have no server access.
What Python version and system dependencies does RealtimeSTT require?
Python 3.11 or newer is required. On Linux, install python3-dev and portaudio19-dev via apt before running pip install. On macOS, install PortAudio via Homebrew first. CUDA support for GPU-accelerated transcription is optional and documented in the repository's installation guide.
Similar open-source tools#
whisper-asr-webservice
Self-hosted speech recognition API built on OpenAI Whisper
code-graph-rag
AI-powered codebase analysis with knowledge graphs
Embabel
Agentic AI framework for the JVM
codebase-memory-mcp
Efficient code intelligence for AI coding agents
Flue Framework
Build powerful, autonomous agents with TypeScript.
DeepSeek TUI
A coding agent that lives in your terminal.