Juicefs

Open source alternative to VAST Data, WEKA and Amazon FSx for Lustre

14.7K stars
Apache-2.0
Active this week
Last commit today

Last verified Nov 13, 2026

Run a POSIX-compatible distributed file system on object storage with Kubernetes, Hadoop, and S3 support for AI and big data workloads.

Visit website GitHub14.7K Deploy on Hostinger

Read the full breakdownWho it is for, how it works, trade-offs, the quick start and the FAQ

who it's for

Who Juicefs is for#

AI and ML engineers running distributed training jobs

JuiceFS provides a shared dataset volume that training workers on multiple nodes read and write concurrently over a standard POSIX mount path. No custom data loaders are needed. D-Robotics measured 7x faster training startup time after switching storage to JuiceFS.

Skip if:

Your training job runs on a single node with local NVMe storage, or your pipeline already reads directly from S3 without needing a filesystem path. JuiceFS adds value at multi-node scale where shared POSIX access matters.

Big data engineers migrating Hadoop workloads to cloud

Existing Hive queries, Spark jobs, and Hadoop tools work unchanged using JuiceFS's Hadoop Java SDK and jfs:// URI scheme. NAVER migrated both Hadoop and Kubernetes AI platforms to JuiceFS and measured 1.7x higher write throughput than HDFS.

Skip if:

Your team is building new pipelines from scratch that read directly from S3, without needing HDFS compatibility. For greenfield workloads, direct S3 access with Delta Lake or Apache Iceberg may be simpler than adding a filesystem layer.

Kubernetes platform teams provisioning shared storage

The JuiceFS CSI Driver creates ReadWriteMany PVCs backed by object storage. Any pod on any node shares the same volume, covering distributed training jobs, multi-stage pipelines, and stateful workloads that need a shared filesystem.

Skip if:

Your workloads only need ReadWriteOnce (single-node access) or ephemeral storage. Standard cloud block volumes or emptyDir are simpler for single-pod access patterns.

Multi-cloud platform teams managing petabyte datasets

JuiceFS presents a single POSIX filesystem across AWS S3, Google Cloud Storage, Azure Blob, and on-premises MinIO at the same time. Lepton AI runs production GPU workloads in three regions across the US and Europe, reporting up to 98% lower cost compared to Amazon EFS.

Skip if:

Your data lives exclusively in one cloud provider and you prefer a fully managed service with vendor support contracts. Amazon FSx for Lustre is simpler for single-AWS workloads that need managed operations.

the problem

The problem it solves#

Building AI training pipelines, running Hadoop workloads, or sharing datasets across Kubernetes pods all require high-throughput shared storage that behaves like a local file system. Object storage (S3, GCS, Azure Blob) is cheap and scalable but presents a non-POSIX interface that breaks existing applications requiring a filesystem path.

Network file systems like Amazon EFS deliver POSIX compatibility but fall far behind in throughput and metadata performance at scale. Enterprise parallel file systems like WEKA or Amazon FSx for Lustre deliver the needed performance, but carry licensing costs in the thousands of dollars per month and tie infrastructure to specific clouds or hardware configurations. Teams running AI model training across hundreds of GPU nodes need storage that keeps pace without those constraints.

how Juicefs solves it

How it solves it#

POSIX-compatible interface over object storage

JuiceFS passed all 8,813 tests in pjdfstest, the standard POSIX filesystem compatibility test suite. Applications that use standard file I/O work without code changes, including tools that rely on rename atomicity, file locking (BSD flock and POSIX fcntl), extended attributes, and mmap.

Pluggable metadata engine

File metadata (names, sizes, permissions, directory structure) is stored separately from data in a user-chosen engine: Redis for high throughput, MySQL or PostgreSQL for durability, TiKV for distributed setups, or SQLite for single-node development. Each option scales independently from the data layer.

Multi-protocol access: POSIX, HDFS, and S3

JuiceFS exposes the same underlying data through three interfaces simultaneously. The POSIX FUSE mount works for any application. The Hadoop Java SDK provides Hadoop 2.x and 3.x compatibility for Hive, Spark, and Flink without modifications to existing jobs. The S3 Gateway gives S3-API access to any tool that speaks S3.

Kubernetes-native persistent storage

The JuiceFS CSI Driver provisions PersistentVolumeClaims with ReadWriteMany access mode, so multiple pods on different nodes share one filesystem volume concurrently. This covers the standard AI training pattern of a dataset volume shared across a GPU job's worker pods.

Data encryption and compression

JuiceFS encrypts data in transit and at rest. Compression uses LZ4 or Zstandard, applied before writing blocks to object storage, reducing storage costs on compressible datasets. Both can be enabled per filesystem at format time.

strengths · trade-offs

Strengths and trade-offs#

Strengths

10x sequential throughput over Amazon EFS and S3FSIn published fio benchmarks, JuiceFS delivers 10x the sequential read and write throughput of Amazon EFS and S3FS. The distributed caching layer absorbs hotspot reads, and the metadata engine handles millions of requests per second at hundreds-of-microseconds latency.
Apache 2.0 license: no licensing fees or restrictionsThe community edition is Apache 2.0 licensed, permitting commercial use, modification, and redistribution. Teams run it in production on their own infrastructure without licensing costs, audit restrictions, or vendor approval.
Production-verified at petabyte scaleMiniMax runs hundreds of petabytes on hybrid cloud. Lepton AI manages billions of files across 1,000-GPU tenant clusters in three regions, reporting up to 98% lower cost than Amazon EFS. D-Robotics stores 120 million files in a single directory for robotics AI training.
Drop-in HDFS replacement with no code changesNAVER switched both Hadoop and Kubernetes AI platforms to JuiceFS and measured 1.7x higher write throughput than HDFS, with zero modifications to existing Hive, Spark, or MapReduce jobs. The jfs:// URI scheme preserves Hadoop convention.

Trade-offs

-Metadata engine must be provisioned and maintained separatelyJuiceFS does not bundle a metadata backend. Redis, MySQL, or PostgreSQL must be provisioned, configured, and backed up as a separate service. Losing Redis metadata without a backup means losing the filesystem index. Teams without existing database operations experience need to account for this operational overhead.
-Small random writes are slower than local NVMeJuiceFS writes data in 4 MiB blocks to object storage, which is efficient for large sequential I/O but adds latency for workloads that do many small random writes, such as databases or transaction logs. Distributed caching helps but adds memory and disk requirements to each client.
-Anonymous usage reporting is on by defaultBy default, JuiceFS reports anonymous metrics (version number only, no user data) to Juicedata. Opting out requires passing --no-usage-report on every mount command, which needs to be included in deployment scripts, Kubernetes CSI configuration, and Docker run commands.

versus alternatives

Juicefs vs alternatives#

JuiceFS vs WEKA

Both JuiceFS and WEKA deliver high-performance shared file storage for AI and HPC workloads, with POSIX compatibility and support for large GPU clusters. WEKA is a proprietary system designed for extreme performance on dedicated hardware or cloud instances; JuiceFS is Apache 2.0-licensed and runs on commodity object storage with a pluggable metadata backend.

Feature	JuiceFS	WEKA
License	Apache 2.0	Proprietary
Data backend	Object storage (S3, GCS, Azure, MinIO, local disk)	Proprietary distributed backend
Metadata	Redis, MySQL, PostgreSQL, TiKV, SQLite	Proprietary
Pricing	Free (self-hosted), cloud service available	Enterprise licensing
POSIX compatibility	Full (8,813 pjdfstest tests passed)	Full

JuiceFS is the better choice for teams that want AI training workloads on existing cloud object storage without paying for enterprise storage licensing, or who need to share storage across public clouds and on-premises infrastructure simultaneously. WEKA offers simpler operations for teams that prefer a fully managed turnkey system and are willing to pay enterprise pricing for it.

JuiceFS vs Amazon FSx for Lustre

Amazon FSx for Lustre is AWS's managed Lustre file system for HPC and ML workloads with sub-millisecond latency and native S3 integration. JuiceFS serves a similar use case but runs across any cloud provider, on-premises, or in hybrid configurations.

Feature	JuiceFS	Amazon FSx for Lustre
License	Apache 2.0	Proprietary (managed service)
Cloud lock-in	None (runs on any S3-compatible backend)	AWS-only
Hadoop compatibility	Yes (Hadoop Java SDK)	No
Kubernetes support	CSI Driver, ReadWriteMany PVCs	FUSE mount only
Cost model	Object storage costs + metadata engine	Per-GB-hour on AWS

JuiceFS avoids AWS vendor lock-in and runs in multi-cloud or hybrid setups without migrating data. Amazon FSx for Lustre is worth considering for workloads already committed to AWS infrastructure that need fully managed operations and AWS support contracts.

JuiceFS vs VAST Data

VAST Data targets enterprise AI-scale storage with a unified file, object, and database layer on proprietary hardware. JuiceFS focuses on the POSIX file system layer, running on top of existing cloud object storage and open source metadata engines. Teams that already have object storage can add a POSIX layer with JuiceFS without hardware procurement or enterprise vendor contracts. VAST Data is worth considering for large enterprises that need a single platform unifying file, object, and database workloads with dedicated hardware and an enterprise support relationship.

install · quick start

Quick start#

bash

Download the JuiceFS client binary for Linux, macOS, or Windows from GitHub Releases: https://github.com/juicedata/juicefs/releases/latest

tech stack · detected from GitHub

What it's built on#

Languages: CGoJavaPython
Cache: Redis

frequently asked

FAQ#

What metadata engines does JuiceFS support?

JuiceFS supports Redis, MySQL, PostgreSQL, TiKV, and SQLite as metadata engines. Redis is recommended for production because it handles millions of metadata requests per second with sub-millisecond latency. For large-scale distributed setups, TiKV removes the single-node capacity limits of Redis. SQLite works for local development and single-node use cases but is not suitable for multi-client production deployments.

Is JuiceFS production-ready?

Yes. JuiceFS has been in production since 2021 and is used by thousands of clients across numerous organizations. Production adopters include MiniMax (hundreds of petabytes on hybrid cloud), Lepton AI (billions of files across 1,000-GPU tenant clusters), D-Robotics (120 million files in a single directory for robotics AI training), and NAVER for unified Hadoop and Kubernetes data access.

How does JuiceFS perform compared to Amazon EFS?

In published benchmarks using fio and mdtest, JuiceFS delivers approximately 10x the sequential throughput of Amazon EFS and S3FS, and significantly higher metadata IOPS. Lepton AI reported up to 98% cost reduction compared to EFS for a comparable workload. The throughput gap depends on caching configuration and network conditions, but JuiceFS consistently outperforms EFS on throughput-bound workloads.

Can JuiceFS run on-premises without cloud object storage?

Yes. JuiceFS supports local disk and self-hosted object storage like MinIO or Ceph RGW as the data backend, allowing a fully on-premises deployment without external cloud dependencies. The metadata engine (Redis, MySQL, etc.) also runs on-premises. The architecture is the same: data in the object store, metadata in the engine, file access through the JuiceFS client.

Does JuiceFS send usage data externally?

By default, JuiceFS reports anonymous usage metrics to Juicedata. Only core metrics like version number are sent; no file data, file names, or user information is included. To opt out, pass --no-usage-report on the mount command. The flag needs to be added to mount scripts, Kubernetes CSI configuration, or Docker run commands to take effect consistently.

also worth a look

Similar open-source tools#

Nvke

Alternative toNVIDIA Run:ai

Kubernetes management plane for NVIDIA AI factory operators.

1,125 GitHub stars
Licence Apache-2.0

Code Server

Alternative toGitHub Codespaces

VS Code in the browser, self-hosted, with no per-seat pricing.

79,830 GitHub stars
Licence MIT

Self Host

Alternative toEasypanel

CLI tool for 93 self-hosted apps with an agent-ready install workflow.

1,584 GitHub stars
Licence Apache-2.0

Keda

Alternative toKedify

Event-driven Kubernetes autoscaling with 70+ scalers and scale to zero

10,717 GitHub stars
Licence Apache-2.0

Duckle

Alternative toFivetran

Self-hosted ETL on DuckDB with 385 connectors and no per-row billing.

1,554 GitHub stars
Licence Apache-2.0

Herdr

Alternative toConductor

Agent runtime that keeps terminals alive across machines

49,868 GitHub stars
Licence Apache-2.0

by category

Self-host Juicefs on a Hostinger VPS

Keep it running 24/7, even when your laptop is off. A Hostinger VPS gives it a server of its own.

01
Choose a plan
Starts from $6.49/mo
02
Apply
Extra 10% off yearly plans
03
Install Juicefs
On a server that stays on
04
Start building
Focus on the app, not the server

Deploy on Hostinger

Inbox Zero

Alternative toSuperhuman

AI email assistant that drafts replies and clears your inbox

12,987 GitHub stars
Licence AGPL-3.0
#Email

rakazo

Alternative toGrok Bot

AI teammates you own: your keys, your model, your machine.

7,869 GitHub stars
Licence Apache-2.0
#AI Agents

SnitchFeed

Live monitoring on LinkedIn, X, Reddit & more. Agentic social listening for startups.

Marketing & Growth

Advertise your product

The projects we add and every new guide. Free, unsubscribe any time.

Find open source alternatives to paid software

JuiceFS vs WEKA

Feature	JuiceFS	WEKA
License	Apache 2.0	Proprietary
Data backend	Object storage (S3, GCS, Azure, MinIO, local disk)	Proprietary distributed backend
Metadata	Redis, MySQL, PostgreSQL, TiKV, SQLite	Proprietary
Pricing	Free (self-hosted), cloud service available	Enterprise licensing
POSIX compatibility	Full (8,813 pjdfstest tests passed)	Full

JuiceFS vs Amazon FSx for Lustre

Feature	JuiceFS	Amazon FSx for Lustre
License	Apache 2.0	Proprietary (managed service)
Cloud lock-in	None (runs on any S3-compatible backend)	AWS-only
Hadoop compatibility	Yes (Hadoop Java SDK)	No
Kubernetes support	CSI Driver, ReadWriteMany PVCs	FUSE mount only
Cost model	Object storage costs + metadata engine	Per-GB-hour on AWS

Who Juicefs is for#

AI and ML engineers running distributed training jobs

Skip if:

Big data engineers migrating Hadoop workloads to cloud

Skip if:

Kubernetes platform teams provisioning shared storage

Skip if:

Multi-cloud platform teams managing petabyte datasets

Skip if:

The problem it solves#

How it solves it#

POSIX-compatible interface over object storage

Pluggable metadata engine

Multi-protocol access: POSIX, HDFS, and S3

Kubernetes-native persistent storage

Data encryption and compression

Strengths and trade-offs#

Strengths

Trade-offs

Juicefs vs alternatives#

JuiceFS vs WEKA

JuiceFS vs Amazon FSx for Lustre

JuiceFS vs VAST Data

Quick start#

What it's built on#

FAQ#

Similar open-source tools#

Open source tools in these categories#

Databases & Storage

Cloud & Hosting

DevOps & CI/CD

AI & Machine Learning

Find open source alternatives to paid software

Who Juicefs is for#

AI and ML engineers running distributed training jobs

Skip if:

Big data engineers migrating Hadoop workloads to cloud

Skip if:

Kubernetes platform teams provisioning shared storage

Skip if:

Multi-cloud platform teams managing petabyte datasets

Skip if:

The problem it solves#

How it solves it#

POSIX-compatible interface over object storage

Pluggable metadata engine

Multi-protocol access: POSIX, HDFS, and S3

Kubernetes-native persistent storage

Data encryption and compression

Strengths and trade-offs#

Strengths

Trade-offs

Juicefs vs alternatives#

JuiceFS vs WEKA

JuiceFS vs Amazon FSx for Lustre

JuiceFS vs VAST Data

Quick start#

What it's built on#

FAQ#

Similar open-source tools#

Open source tools in these categories#

Databases & Storage

Cloud & Hosting

DevOps & CI/CD

AI & Machine Learning

Find open source alternatives to paid software