Who turbovec is for#
Python developers building RAG pipelines on memory-constrained servers
turbovec reduces the RAM footprint of a 10 million vector corpus from 31 GB to 4 GB, making it feasible to run embedding search on a standard cloud VM alongside the application. The pip install turbovec extras for LangChain and LlamaIndex replace the default in-memory stores with no API changes.
Skip if:
Your corpus fits comfortably in RAM as float32 and you have no memory budget pressure. At small corpus sizes, uncompressed in-memory stores carry lower setup overhead and no calibration step.
Engineers building air-gapped or privacy-sensitive AI applications
turbovec is a local library with no external service calls. Paired with a self-hosted embedding model, it provides a complete vector search stack that never sends data outside the machine or VPC. This fits regulated industries, on-device AI, and enterprise environments with strict data residency requirements.
Skip if:
Your threat model allows data to leave the machine and you prefer a managed API with SLA guarantees. Pinecone or Weaviate Cloud offer higher operational maturity and built-in redundancy for teams that do not need air-gapped deployment.
Rust developers embedding vector search in native applications
The Rust crate provides the same TurboQuantIndex and IdMapIndex types as the Python bindings, with a cargo add turbovec install path. This lets Rust applications embed fast quantized vector search without a Python runtime dependency or a sidecar service.
Skip if:
Your application is not written in Rust or Python. turbovec does not currently expose an HTTP or gRPC API, so embedding it in other runtimes requires FFI bindings that are not yet officially provided.
Teams replacing framework default vector stores in existing pipelines
The turbovec extras for LangChain, LlamaIndex, Haystack, and Agno install drop-in replacements for each framework's default in-memory document or vector store. Swapping the import preserves retriever wiring, pipeline semantics, and persistence behavior while adding TurboQuant compression.
Skip if:
Your pipeline already persists to a dedicated vector database with metadata filtering and multi-tenancy. turbovec is a single-index library without a full document store or metadata layer.
The problem it solves#
Vector search at scale runs into two walls: memory and latency. A corpus of 10 million float32 embeddings takes 31 GB of RAM to hold without compression. Most teams avoid this by offloading to a managed vector database, but that means data leaves the machine, query latency grows with network round-trips, and the monthly bill scales with index size and query volume.
For teams building RAG pipelines where privacy matters, where data cannot leave the VPC, or where the embedding corpus is large enough that RAM is a genuine constraint, the pain is real: the self-hosted options with acceptable recall either require a separate training phase (which means rebuilds as the corpus grows) or offer search throughput that trails managed services by a wide margin.
How it solves it#
TurboQuant 2-bit and 4-bit compression
Compresses float32 vectors using Google Research's TurboQuant algorithm, a data-oblivious quantizer that needs no per-dataset training. A 10 million vector corpus that occupies 31 GB as float32 fits in 4 GB at 4-bit, with recall competitive with FAISS IndexPQ across all tested embedding dimensions.
SIMD search kernels for ARM and x86
Hand-written NEON SDOT/SMMLA kernels on ARM and AVX-512 VNNI on x86 beat FAISS IndexPQFastScan by an average of 3.4x at 4-bit and 23% at 2-bit across all benchmark configurations. AVX2 and scalar fallback kernels cover older hardware automatically.
Online ingest without a training phase
Vectors are indexed immediately on add() with no train step, no parameter tuning, and no index rebuilds as the corpus grows. Single-vector inserts land in 6.3 to 19.7 microseconds depending on configuration, which is 7.6 to 13.9 times faster than a FAISS single add.
Incremental crash-safe persistence
The sync(path) call persists only what changed since the last sync in one fsync with an atomic rename, crash-safe at any byte. Appending a small batch or removing an entry costs milliseconds regardless of total index size. Full snapshots are available via write() and load().
Filtered search with no recall penalty
Pass an id allowlist or slot bitmask to search() and the SIMD kernel honors it at 32-vector block granularity: blocks with no allowed slots are short-circuited before any scoring work. Selective allowlists avoid most SIMD cost rather than paying it and discarding results afterward.
Drop-in framework integrations
Replaces the default in-memory vector stores for LangChain (InMemoryVectorStore), LlamaIndex (SimpleVectorStore), Haystack (InMemoryDocumentStore), and Agno (LanceDb) via extras installs: pip install turbovec[langchain], pip install turbovec[llama-index], and so on. Same public API, same persistence semantics.
Strengths and trade-offs#
Strengths
- No training phase or index rebuildsMost quantization indexes require a training pass over representative data before ingest can begin. TurboQuant's data-oblivious rotation makes the coordinate distribution predictable regardless of input, so turbovec starts indexing from the first vector and never requires rebuilds as the corpus grows.
- MIT license, fully air-gapped deploymentThe MIT license allows commercial use with no attribution requirements and no AGPL network-clause restrictions. Because turbovec is a local library with no managed service component, it can run in air-gapped environments where no data leaves the machine or VPC.
- O(1) id-based deletion via IdMapIndexThe IdMapIndex class provides stable uint64 external ids that survive deletes. Removal is O(1) via swap-and-pop plus id-map bookkeeping, landing at 0.44 to 1.22 microseconds per op. The equivalent FAISS operation repacks stored codes on every call, reaching up to 1.02 seconds per single remove at 100K vectors.
- Published benchmarks with reproducible scriptsRecall and speed benchmarks cover OpenAI embeddings at d=1536 and d=3072 and GloVe at d=200, with full results published as JSON in the repository. The comparisons use FAISS IndexPQ with LUT256 and nbits=8 as the baseline, a stronger standard than the one used in the original TurboQuant paper.
Trade-offs
- -Young project: first released March 2026The repository was created in March 2026 and reached 1.0 shortly after. With 15 open issues and a short public history, the library has not been through the production incident cycles that longer-lived indexes like FAISS have. Teams with strict stability requirements should evaluate it against a non-critical workload first.
- -No distributed or multi-node indexturbovec is a single-process library, not a distributed database. There is no built-in sharding, replication, or cluster mode. Corpora that do not fit in a single machine's RAM require a separate partitioning strategy. Managed services like Pinecone or Weaviate Cloud handle this transparently.
- -Calibration step needed at low embedding dimensionsAt low embedding dimensions (d=200 and below), TurboQuant's asymptotic Beta assumption is looser, and uncalibrated recall can trail FAISS at 2-bit. The TQ+ calibration step (one call to index.calibrate(sample) with roughly 1024 representative vectors) recovers the deficit, but it is an extra step that higher-dimensional use cases do not require.
turbovec vs alternatives#
turbovec vs Pinecone
Both provide vector similarity search for embedding-based retrieval, but they take opposite approaches to deployment. Pinecone is a fully managed cloud service with serverless pricing; turbovec is a local MIT-licensed library that runs in-process with no external dependency.
| Feature | turbovec | Pinecone |
|---|---|---|
| License | MIT | Proprietary |
| Deployment | Local in-process library | Managed cloud only |
| Self-hosting | Yes | No |
| Training phase | None (data-oblivious) | None (serverless) |
| Distributed index | No | Yes |
| Pricing | Free (self-hosted) | Paid (serverless billing) |
turbovec is the better choice when data must not leave your infrastructure, when corpus size makes managed per-query pricing uneconomical, or when you need O(1) vector deletion without index repacking. Pinecone is still the better choice for teams that need a distributed index across billions of vectors, built-in metadata filtering with a complex query syntax, and zero infrastructure management. At production scale with multi-region replication requirements, Pinecone's managed architecture carries less operational overhead.
turbovec vs Qdrant
Qdrant is an open source vector database with a Docker deployment path and a hosted cloud service. turbovec is an in-process library, not a standalone service. The two tools serve overlapping use cases but from different architectural positions.
| Feature | turbovec | Qdrant |
|---|---|---|
| Deployment | In-process library | Standalone server (Docker) |
| Self-hosting | Yes | Yes |
| REST/gRPC API | No | Yes |
| Filtering | Id allowlist / bitmask | Rich payload query language |
| Quantization | TurboQuant 2-bit / 4-bit | Multiple quantization methods |
turbovec is the better choice when you need an in-process library that adds compressed vector search to an existing Python or Rust process without running a sidecar service. Qdrant is the better choice when you need a full vector database with a language-agnostic HTTP API, rich metadata filtering across multiple collections, and managed cloud hosting. Teams that already run Qdrant as a service gain little by switching to turbovec; the tools serve different architectural needs.
Install and self-host#
Install turbovec from PyPI for Python or from crates.io for Rust; no external service or database setup is required.
```bash
pip install turbovec
cargo add turbovec
```What it's built on#
- Languages
- CPythonRust
- Search
- FAISS
FAQ#
Is turbovec free to use commercially?
Yes. turbovec is MIT licensed, which means you can use it in commercial products, modify it, and distribute it without restriction. There is no managed service, no per-query fee, and no separate commercial license required. You pay only for the infrastructure you run it on.
How does turbovec compare to FAISS for vector search speed?
In benchmarks on 100K OpenAI vectors at k=64, turbovec's SIMD kernels beat FAISS IndexPQFastScan by an average of 3.4x at 4-bit and 23% at 2-bit on both ARM and x86 hardware. Recall is also competitive: calibrated TurboQuant (TQ+) beats FAISS at R@1 on three of four configurations at OpenAI dimensions and on both bit widths for GloVe d=200. The key operational difference is that turbovec requires no separate training step, while FAISS IndexPQ uses k-means++ for codebook training before any ingest can begin.
Does turbovec support filtered or hybrid vector search?
Yes. Pass an id allowlist or slot bitmask to search() and the kernel filters inside the SIMD scoring loop at 32-vector block granularity. Blocks with no allowed slots are short-circuited before any scoring work, so selective allowlists avoid most SIMD cost. This enables hybrid retrieval: an external system such as SQL or BM25 narrows the candidate set, and turbovec reranks within it. The output length is min(k, n_allowed), so you always get exactly the right number of results without over-fetching.
Can turbovec replace Pinecone or Qdrant?
It can replace them for single-node workloads where data must stay local. turbovec provides a local MIT-licensed index with TurboQuant compression, fast SIMD search, and O(1) id-based deletion. What it does not provide is the distributed architecture, managed replication, and multi-tenant metadata filtering that hosted services like Pinecone or Qdrant Cloud offer. For privacy-sensitive or cost-constrained use cases on a single machine, turbovec is a strong alternative to both.
Which Python frameworks does turbovec integrate with?
turbovec provides drop-in extras for LangChain (replaces InMemoryVectorStore), LlamaIndex (replaces SimpleVectorStore), Haystack (replaces InMemoryDocumentStore), and Agno (replaces LanceDb). Each extras install follows the pattern pip install turbovec[langchain]. The replacements expose the same public API and persistence semantics as the originals: swap the import and keep your pipeline wiring unchanged.
Similar open-source tools#
Mengram
AI memory for Claude Code with auto-save across sessions
Qdrant
Self-hosted vector database for AI similarity search and RAG
Weaviate
AI-native vector database for semantic search and AI apps
cognee
Persistent memory for AI agents across sessions
CocoIndex
Incremental data framework for AI agents.
RAG-Anything
Comprehensive multimodal document processing framework