Anserini

Open source alternative to Elastic (Elasticsearch)

1.2K stars
Apache-2.0
Active this week
Last commit 1 day ago

Last verified Nov 15, 2026

Search and index documents using Apache Lucene for reproducible information retrieval research, bridging academic benchmarks and production search.

Visit website GitHub1.2K

Read the full breakdownWho it is for, how it works, trade-offs and the FAQ

who it's for

Who Anserini is for#

IR researchers running standard benchmark experiments

Anserini provides prebuilt indexes for MS MARCO, BEIR, and TREC collections with documented reproduction scripts, so researchers can test a new retrieval model against standard datasets and verify that their reported numbers match the reference baselines independently.

Skip if:

Your work focuses on neural re-ranking with no sparse retrieval component. Pyserini extends to some dense retrieval cases, but Anserini's strongest value is sparse and hybrid retrieval experiments on standard IR benchmarks.

Engineering teams evaluating search quality before a production deployment

Teams migrating from a commercial search deployment can use Anserini to test retrieval effectiveness on their documents against established academic baselines. The shared Lucene core means BM25 behavior is consistent between Anserini experiments and a production Elasticsearch or OpenSearch index.

Skip if:

You need a full production search service with REST query APIs, distributed indexing, and monitoring out of the box. Anserini is not a drop-in replacement for Elasticsearch: it is a research toolkit, not a managed search server.

Students and educators teaching information retrieval

The fatjar install and prebuilt indexes make Anserini accessible in a course environment without requiring students to compile from source. The SIGIR 2017 and JDIQ 2018 papers provide peer-reviewed methodology references that connect coursework to published IR research.

Skip if:

Your course uses Python exclusively and has no Java requirement. Pyserini is the recommended entry point for Python-first curricula and covers most of the same retrieval experiments.

the problem

The problem it solves#

Information retrieval research depends on reproducibility: a retrieval model published in a paper should produce the same rankings when anyone else runs it. In practice, most academic IR systems are written in bespoke Python or C++ code, tied to a specific dataset version, and impossible to run outside the original lab. This makes it hard to build on prior work, compare new models fairly, or trust that a paper's reported numbers reflect the actual system.

The pain sharpens when moving from research to production. Lucene powers the majority of commercial and open source search infrastructure, yet most IR research runs on retrieval engines with no connection to Lucene. Teams that want to bring a research-grade retrieval model into a Lucene-based production system often have to reimplement the whole thing from scratch, which reintroduces the reproducibility problem the original research was meant to solve.

how Anserini solves it

How it solves it#

Self-contained fatjar install

Anserini ships as a prebuilt fatjar with all Lucene 10.5 dependencies bundled. Download it once and run Lucene indexing and search without compiling source or managing a Maven build. The same jar works on any JVM-capable host, so there is no environment-specific setup required.

Prebuilt indexes for standard IR benchmarks

Ready-to-use prebuilt indexes cover common IR test collections including MS MARCO, BEIR, and TREC datasets. Researchers can reproduce published retrieval experiments without re-indexing from raw collections, which saves hours of preprocessing time for each experimental run.

Lucene 10 retrieval primitives

Direct access to Apache Lucene 10's indexing and search API, including BM25 and query-expansion retrieval. The same code path runs both controlled research experiments and production Lucene stacks, so a model proven on a benchmark transfers cleanly to a deployed Lucene application.

Pyserini Python interface

Most Anserini retrieval functionality is available through Pyserini, a Python library wrapping the Java toolkit. Researchers who prefer Python can run the same benchmarks and obtain the same rankings as the Java path, without writing any Java code.

Documented reproduction scripts with reference numbers

Every experiment in the codebase ships with a documented reproduction script and reference effectiveness numbers. Contributors and reviewers can verify that a code change has not silently broken a retrieval model's accuracy, which is the core commitment of the reproducibility-first design.

strengths · trade-offs

Strengths and trade-offs#

Strengths

Apache-2.0 permissive licenseAnserini is Apache-2.0 licensed, an OSI-approved permissive license. You can run it in commercial applications, modify the source, and distribute modified versions without triggering copyleft obligations and without paying any licensing fee. Unlike Elasticsearch's commercial license, there are no restrictions on research or internal use.
Peer-reviewed academic pedigreeAnserini has publications at SIGIR 2017 and in the Journal of Data and Information Quality (2018), and continues active development with 1,238 GitHub stars, 771 forks, and a last commit in November 2026. The castorini research group has sustained reproducibility benchmarks for over a decade, making the codebase a credible academic reference.
Same Lucene core as production search platformsBecause Anserini runs on the Apache Lucene library that also powers Elasticsearch and OpenSearch, retrieval models prototyped in Anserini benchmarks can migrate into production Lucene infrastructure without reimplementing the retrieval logic. The overlap in the retrieval engine means benchmark results translate directly.

Trade-offs

-No production search API or cluster modeAnserini is a command-line and programmatic research toolkit, not a search server. It has no REST API, no index replication, and no distributed cluster mode. Teams that need a queryable HTTP endpoint, real-time document updates at scale, or multi-node search must use Elasticsearch or OpenSearch instead.
-Java core required for full functionalityAnserini's command-line tools and source-level customization require a JDK. The Pyserini Python interface covers most retrieval features, but researchers who need Java-only tools or want to modify the retrieval pipeline will need a Java development environment. Teams with no Java experience may find the Maven build setup takes time.
-Upstream evaluation tooling restructured in 2026The README notes that the eval repository's topics-and-qrels directory structure was refactored in August 2026, breaking consumers relying on stable paths from the master branch. Teams building automated pipelines on top of Anserini should pin to specific commits rather than following master to avoid unexpected breakage.

versus alternatives

Anserini vs alternatives#

Anserini vs Elasticsearch

Both tools run on Apache Lucene, but they address completely different use cases. Elasticsearch is a commercial managed search and analytics platform for production workloads. Anserini is a research toolkit for reproducible information retrieval experiments. The overlap is that both expose Lucene retrieval, so models proven in Anserini benchmarks translate to production Elasticsearch indexes without a retrieval logic rewrite.

Feature	Anserini	Elasticsearch
License	Apache-2.0	Commercial (proprietary)
Self-hosting	Yes (single process)	Yes (cluster setup)
REST API	No	Yes
Distributed search	No	Yes
Managed cloud	No	Yes (Elastic Cloud)
Primary purpose	IR research and benchmarking	Production search and analytics
Install method	Fatjar (JVM only)	Package manager or Docker

Anserini is the right tool when the goal is reproducible IR research: running standard benchmarks like MS MARCO or BEIR, comparing retrieval models against published baselines, and producing results that other researchers can replicate with the same code. Apache-2.0 licensing means there are no cost barriers to running experiments or publishing results.

Elasticsearch is worth using when you need a full production search service: REST query APIs, real-time document indexing, distributed cluster management across multiple nodes, and a commercially supported cloud offering. For teams that want to benchmark retrieval quality using Anserini's prebuilt indexes before committing to an Elasticsearch deployment, that transition is straightforward because both tools share the same Lucene retrieval engine.

tech stack · detected from GitHub

What it's built on#

Languages: JavaJuliaPython

frequently asked

FAQ#

How do I install Anserini?

The simplest path is the fatjar: the README describes downloading a prebuilt JAR via curl, with no Maven build or source checkout required. The installation guide at docs/installation-fatjar.md in the repo covers the exact command and steps. For a full development environment that supports source changes and all tests, docs/installation-dev.md covers the Maven-based path. Most retrieval features are also available through Pyserini for researchers who prefer Python.

Does Anserini require Java?

Yes, the core toolkit is Java. The fatjar requires a JVM to run, but no JDK or Maven is needed for basic retrieval use. For source-level development, a JDK and Maven are required. If you prefer Python, Pyserini wraps Anserini's Java core in a Python library and covers most retrieval features, so many researchers interact with Anserini entirely through Python without writing any Java.

What is the difference between Anserini and Elasticsearch?

Both tools use Apache Lucene, but they serve completely different purposes. Elasticsearch is a production search and analytics platform with REST APIs, distributed clustering, and commercial managed hosting. Anserini is a research toolkit for reproducible IR experiments: no REST API, no clustering, no managed tier. Teams running IR benchmarks and experiments use Anserini; teams that need a deployable search service with query APIs use Elasticsearch or OpenSearch.

What benchmarks does Anserini support?

Anserini includes prebuilt indexes and reproduction scripts for major IR test collections including MS MARCO, BEIR, and TREC datasets. The repo's documentation provides reference effectiveness numbers so researchers can verify that a reproduction matches the published baselines, which is central to the reproducibility-first design. The onboarding path at docs/start-here.md in the repo walks through the first experiments.

Is Anserini suitable for a production search deployment?

No. Anserini is a command-line and programmatic research toolkit, not a search server. It has no REST API endpoint, no index replication, and no cluster mode. For production search, Elasticsearch or OpenSearch are the natural options. Because all three share the same Lucene retrieval engine, a retrieval model prototyped in Anserini can be carried into a production Lucene deployment without reimplementing the core logic.

also worth a look

Similar open-source tools#

Rerun

Alternative toFoxglove

Log, visualize, and query multimodal physical AI data.

11,791 GitHub stars
Licence Apache-2.0

Aleph

InData & Analytics

Python CLI for satellite imagery, terrain, buildings and Street View.

881 GitHub stars
Licence MIT

Unlimited-OCR

Alternative toGoogle Document AI

Document OCR with unlimited page depth, MIT licensed, self-hosted.

31,159 GitHub stars
Licence MIT

Dolt

Alternative toPlanetScale

Git-native SQL database with branching, merging, and row-level diffs.

24,900 GitHub stars
Licence Apache-2.0

Lean

Alternative toQuantConnect

Algorithmic trading engine for backtesting and live trading

22,512 GitHub stars
Licence Apache-2.0

Marker

Alternative toUnstructured.io

Convert documents to clean markdown and JSON with open source AI.

40,910 GitHub stars
Licence Apache-2.0

by category

Open source tools in these categories#

121

13,025 GitHub stars
Licence AGPL-3.0
#Email

rakazo

Alternative toGrok Bot

AI teammates you own: your keys, your model, your machine.

8,291 GitHub stars
Licence Apache-2.0
#AI Agents

SnitchFeed

Live monitoring on LinkedIn, X, Reddit & more. Agentic social listening for startups.

Marketing & Growth

Advertise your product

The projects we add and every new guide. Free, unsubscribe any time.

Find open source alternatives to paid software

Anserini vs Elasticsearch

Feature	Anserini	Elasticsearch
License	Apache-2.0	Commercial (proprietary)
Self-hosting	Yes (single process)	Yes (cluster setup)
REST API	No	Yes
Distributed search	No	Yes
Managed cloud	No	Yes (Elastic Cloud)
Primary purpose	IR research and benchmarking	Production search and analytics
Install method	Fatjar (JVM only)	Package manager or Docker

Who Anserini is for#

IR researchers running standard benchmark experiments

Skip if:

Engineering teams evaluating search quality before a production deployment

Skip if:

Students and educators teaching information retrieval

Skip if:

The problem it solves#

How it solves it#

Self-contained fatjar install

Prebuilt indexes for standard IR benchmarks

Lucene 10 retrieval primitives

Pyserini Python interface

Documented reproduction scripts with reference numbers

Strengths and trade-offs#

Strengths

Trade-offs

Anserini vs alternatives#

Anserini vs Elasticsearch

What it's built on#

FAQ#

Similar open-source tools#

Open source tools in these categories#

Data & Analytics

Developer Tools

Find open source alternatives to paid software

Who Anserini is for#

IR researchers running standard benchmark experiments

Skip if:

Engineering teams evaluating search quality before a production deployment

Skip if:

Students and educators teaching information retrieval

Skip if:

The problem it solves#

How it solves it#

Self-contained fatjar install

Prebuilt indexes for standard IR benchmarks

Lucene 10 retrieval primitives

Pyserini Python interface

Documented reproduction scripts with reference numbers

Strengths and trade-offs#

Strengths

Trade-offs

Anserini vs alternatives#

Anserini vs Elasticsearch

What it's built on#

FAQ#

Similar open-source tools#

Open source tools in these categories#

Data & Analytics

Developer Tools

Find open source alternatives to paid software