Billion Context

Open source alternative to Compresr

4.6K stars
MIT with attribution requirement
Active this week
Last commit today

Last verified Nov 8, 2026

Run a context-compression proxy between any AI coding agent and its model API, cutting token costs up to 5x and enabling month-long sessions.

Visit website GitHub4.6K

Read the full breakdownWho it is for, how it works, trade-offs, the quick start and the FAQ

who it's for

Who Billion Context is for#

Developers running multi-hour AI coding sessions

billion-context keeps a session alive across days of work without breaking context. If you regularly hit the edge of your model's context window on complex refactors, long debugging sessions, or large codebases, the proxy keeps the conversation coherent without requiring a manual restart.

Skip if:

Your sessions are short and self-contained (under an hour per task). billion-context's value is for extended runs where context accumulates; for short sessions, the overhead of a local proxy exceeds the benefit.

Teams with high monthly model API costs

At 5x fewer tokens for the same session quality, teams paying several hundred dollars per month on API fees for coding agents can cut those costs significantly. The npm install and local proxy are free; the savings come entirely from reduced token usage on the model API you are calling.

Skip if:

Your team is on a fixed-price or unlimited plan for your model API. If token volume is not what drives your costs, the compression benefit does not translate to a cost saving.

Agent pipeline developers building long-horizon tasks

For agentic pipelines where a single task may involve hundreds of tool calls and thousands of messages, billion-context prevents the pipeline from stopping at a context boundary mid-task. The proxy is transparent to the agent; no code changes are needed in the pipeline itself.

Skip if:

Your pipeline already manages context windows programmatically and reinitializes agents between subtasks. billion-context is most useful for single-session continuity, not for distributed multi-agent systems where context is managed at a higher level.

the problem

The problem it solves#

Long AI coding sessions hit a wall. Each model charges per token, and context windows run out. Once the window overflows the session breaks or degrades badly, forcing the user to start fresh and reload all the accumulated project context. Teams running intensive agent sessions routinely pay hundreds of dollars in API fees for a single complex task, with no guarantee the session survives long enough to complete it.

Existing options leave gaps. Provider-side summarization is opaque, non-reversible, and often drops critical details. RAG approaches require extra infrastructure and return imprecise context. Truncation just cuts history. None of them let the agent decide what to retain with full fidelity.

how Billion Context solves it

How it solves it#

Transparent proxy for any AI coding agent

Intercepts and rewrites Anthropic and OpenAI API streams, so billion-context works with any client that calls those APIs: Claude Code, Codex, OpenCode, aider, Gemini CLI, and 20+ others. No model switching, no API key changes, no prompt engineering.

Model-driven incremental compression

Compression is not a hard truncation. The proxy injects four context-management tools into the conversation; the model calls `compress` itself as context grows, folding a message range into a detailed summary. Ranges can be decompressed on demand when exact details matter again, and `search_context` does keyword search across compressed and visible messages.

Prefix-cache compatibility

Compression keeps the cache prefix intact. A healthy session maintains a 95-97% prefix-cache hit rate, and the compression process itself costs under 2% of that. Prefix-cache misses are a primary cost amplifier in long agent sessions; preserving the prefix is what makes the token savings add up.

Three integration modes

Install as a native plugin inside a supported agent client (Claude Code, OpenCode, pi, dsh, kimi, hermes, zcode), use the `bili <client>` launcher to start both proxy and agent together, or point the client's baseURL at the `/bili/` URL prefix. All three routes land in the same compression pipeline.

Independent third-party benchmark results

Third-party 1,500-item benchmarks rank billion-context first on ContextBench (0.3097 Utilized F1, 58.3% lower cost than full context), 16.6% higher recall than the best RAG approach on LongMemEval-S (79.20% vs 62.60%), and 100% survival on the 1.48M-token MemoryAgentBench task where full context fails all 199/199 runs.

strengths · trade-offs

Strengths and trade-offs#

Strengths

No per-token fees for the proxy layerUnlike Compresr and similar commercial context-compression services that charge usage-based fees for their APIs, billion-context runs locally at no marginal cost. The proxy is a local process; your costs reduce to your own server costs and the actual model API calls, which are smaller because of the compression.
Agent compatibility across 20+ clientsThe proxy handles both Anthropic and OpenAI wire formats, which covers the vast majority of AI coding agents in active use. Dedicated launchers, native plugins, and cert-MITM modes expand coverage to closed-binary clients (GitHub Copilot CLI, Amp, Kiro) that have no plugin seam.
Reversible, model-guided compressionUnlike hard truncation or opaque provider summarization, billion-context lets the model decide what to compress and fold it into a summary that the model can decompress later when exact details are needed. This preserves context fidelity better than losing it outright.
Documented production-scale resultsThe project publishes a research paper documenting 4.5 months of production use: 174,327 model calls, 18.76B cumulative input tokens, zero context-window violations on 204,800-token models, and marathon sessions of 8,584 to 12,049 calls. This is documented longitudinal operation, not a lab benchmark.

Trade-offs

-MIT license with mandatory attribution for user-facing productsMIT with an added attribution clause: use, modify, deploy, and self-host for any purpose including commercial. Products or services with a user-facing surface must display visible credit on the home page, docs, or About/Credits page stating the product uses billion-context with a link to the repository. Server-side or embedded use with no user-facing surface needs only a notice in shipped documentation. Reselling it as a managed service is not permitted.
-495 open issues for a project that opened in August 2026The repo reached 4,625 stars quickly after opening in August 2026. With 495 open issues, bug reports and edge cases are actively accumulating. The project shows recent pushes, but users running production workloads should expect rough edges in client integrations and test their specific agent client before committing.
-Cert-MITM mode requires trusting a local CAFor closed-binary agent clients with no plugin seam, billion-context uses certificate-based MITM proxying. This requires installing a local CA certificate, which some security policies prohibit in team or enterprise environments. Clients with native plugin support (Claude Code, OpenCode, pi, dsh, kimi, hermes, zcode) avoid this requirement entirely.

versus alternatives

Billion Context vs alternatives#

Billion Context vs Compresr

Compresr is a commercial context-compression API for LLM pipelines with usage-based pricing. It is designed as a managed service you call from your own code, abstracting the compression logic into a hosted API endpoint.

Billion Context takes a different approach: it is a local proxy you install once that sits between your agent client and the model API, with no changes to your calling code. You point your agent's base URL at the proxy; it handles compression transparently.

Price model

Compresr charges per use of its compression API. Billion Context is free to run; there is no per-call fee for the proxy layer itself. Your costs are only the reduced number of tokens sent to the actual model API, which is where the savings come from.

Self-hosting and code access

Billion Context runs entirely on your machine or server. You can inspect the compression code, modify the acp-kernel behavior, and update without waiting for a vendor. Compresr is a hosted service with no access to the compression implementation.

Integration surface

Compresr integrates at the code level: your application calls its API. Billion Context integrates at the network level: the proxy intercepts and rewrites model API traffic. This means billion-context works with any existing agent client without code changes, while Compresr requires explicit integration work in your application or pipeline.

When Compresr fits better

Compresr is simpler to integrate for teams that already have application-level code calling LLM APIs and want a drop-in compression middleware. It requires no local proxy management and no network interception.

When Billion Context fits better

Billion Context is better suited for individual developers or teams using off-the-shelf AI coding agents (Claude Code, Codex, OpenCode, aider) where there is no application-level code to modify. It is free to run and open source, with detailed benchmark results published in the project's research paper.

install · quick start

Quick start#

bash

Install billion-context globally with npm to get the `bili` command for the context-compression proxy.

```bash
npm install -g billion-context --prefix=~/.local
```

tech stack · detected from GitHub

What it's built on#

Languages: JavaScriptTypeScript
Databases: MySQL
Tooling: esbuild

frequently asked

FAQ#

Does billion-context work with Claude Code?

Yes. Run bili plugin install claude for a native integration that manages its own settings block and session-owned proxy, or use bili claude as a launcher that starts both the proxy and Claude Code together. The native mode is documented in the README as issue #964.

What does the attribution requirement in the license mean for self-hosters?

billion-context uses an MIT license with an added attribution clause. You may use, modify, deploy, and self-host it for any purpose including commercial and team use. If you use it in a product or service with a user-facing surface (a web app, desktop tool, or anything a user interacts with), you must display a visible credit on the home page, documentation, or an About/Credits page stating the product uses billion-context and linking to the GitHub repository. Server-side or embedded use with no user-facing surface satisfies the requirement by carrying the notice in shipped documentation only.

How does billion-context compare to built-in model context summarization?

Provider-side context summarization is opaque and non-reversible; once the provider summarizes history, the original is gone and you cannot restore it. billion-context keeps summaries as explicit, decompressible ranges: the model can call decompress to restore a range if exact details are needed again. It also exposes search_context for keyword search across both compressed and visible history, and acp_status for a context-usage overview.

What is the expected prefix-cache hit rate during a session?

A healthy session maintains a 95-97% prefix-cache hit rate, with compression overhead itself under 2% of that. If the rate drops, the README documents the typical causes in order: upstream cache TTL expiry, a model switch mid-session, a billion-context bug, or unknown causes. Cache health is visible during a session via the /acp and /acp-cache commands.

Does billion-context support OpenAI-compatible APIs beyond the official OpenAI endpoint?

Yes. The proxy rewrites both Anthropic and OpenAI wire format streams. Any model API that uses either format is supported: Anthropic-compatible, OpenAI-compatible, and local models served with an OpenAI-compatible endpoint. The client table in the README lists certified integrations; for clients using other formats, the launcher or URL prefix route applies.

also worth a look