Who Duckle is for#
Data engineers replacing hosted Fivetran or Airbyte
Teams paying per-row or per-connector fees on hosted ETL can move the same pipelines to Duckle, which covers the same source and destination surface with no usage-based billing. The visual canvas matches what connector-based teams already know, and the Python API is available for pipeline-as-code workflows.
Skip if:
If your organization relies on Fivetran's managed reliability guarantees, SLA, and support contracts, Duckle's beta status and self-managed operations are not equivalent today.
Teams building on the DuckDB ecosystem
Duckle sits in the ingestion and transformation layer alongside tools like dlt and dbt, reading DuckLake tables, MotherDuck, the Quack protocol, and community extensions. Teams already using DuckDB can adopt Duckle as the visual and orchestration layer without adding a new engine.
Skip if:
If you only need SQL transformation inside a warehouse you already have, dbt alone may be sufficient. Duckle adds value when extraction or loading are also part of the problem.
Organizations with data-residency or compliance requirements
Air-gapped, on-premises, and regulated environments where data cannot leave controlled infrastructure benefit from Duckle's no-telemetry, no-account, no-vendor-cloud architecture. Deployment targets like EKS or Azure with no outbound calls from Duckle satisfy most data-residency policies.
Skip if:
If your compliance requirement is SOC 2 or ISO certification of the tooling vendor, a managed service with formal certifications is more appropriate than a self-managed open-source deployment.
Teams migrating from Talend Open Studio
Talend Open Studio reached end of life on January 31, 2024, and its free downloads were withdrawn. Duckle covers similar ground: a canvas of sources, transforms, and destinations, a visual mapper for joins and expression mapping, and scheduled or headless execution. Pipelines compile to SQL on DuckDB rather than generated Java, and the whole tool ships as one binary.
Skip if:
Teams with extensive custom Talend components written in Java will need to reimplement those in Duckle's Python or SQL API.
The problem it solves#
Commercial ETL and data integration platforms charge for what they move. Fivetran bills per row at the source. Airbyte Cloud adds per-connector costs on its hosted tier. dbt Cloud adds per-seat pricing on top of the warehouse you already pay for. Teams running high-volume pipelines or scaling from a startup to production often hit billing cliffs that make hosted platforms unworkable, and building custom pipelines in Airflow or Spark requires substantial engineering overhead.
The harder problem is data custody. When pipelines run on a vendor's cloud, schemas, credentials, and query logs pass through infrastructure outside your control. For teams with data-residency requirements, internal security policies, or regulated data, that trade is not acceptable. The practical workaround is usually a fragile mix of scripts, cron jobs, and undocumented shell commands that outlives whoever wrote them.
How it solves it#
Visual canvas with SQL inspection on every node
Drag sources, transforms, validators, and sinks onto a canvas and wire them together. Every node compiles to readable SQL visible in its Plan tab, so no step is a black box. The visual Map editor joins a main input to up to three lookups with per-output typed expressions and an inline filter.
385 components across 116 sources and 72 destinations
Ships with 385 components covering files, SQL databases, warehouses, lakehouses, object storage, streaming brokers, SaaS REST/GraphQL APIs, NoSQL, and vector databases. The list spans Parquet, Iceberg, Delta, Snowflake, BigQuery, Postgres, MongoDB, Salesforce, Stripe, and more, all available at install time.
On-device AI assistant with no API key
Duckie runs Qwen 2.5 Coder via llama.cpp in-process, so describing a pipeline in plain English never sends your schema or data to a third party. Ask for a pipeline in natural language; Duckie streams a valid pipeline and drops positioned nodes onto the canvas. An MCP server also lets external AI agents like Claude or Cursor list, generate, validate, and run pipelines.
dbt on DuckDB inside the same studio
Duckle runs dbt projects on DuckDB with a visual GUI, using a fast build engine by default and falling back to dbt-core when needed. Build, test, and schedule dbt models without leaving the studio or setting up a separate runtime.
Headless server deployment with audit trail
duckle-runner serve runs pipelines headless on a schedule in Docker, on EC2, or on Kubernetes, with a web console, roles, and an audit trail. Every pipeline is one file in git, so it can be diffed, branched, and reviewed like application code.
Change data capture and incremental loading
Supports CDC with upsert and delete propagation; 100,000 changes can be mirrored into DuckDB in about 1.7 seconds. Watermark incremental loading reads only rows past the saved watermark on subsequent runs, reducing load on source systems.
Strengths and trade-offs#
Strengths
- No per-row, per-connector, or per-seat billingDuckle is Apache-2.0 licensed and free to run on any infrastructure, with no usage-based fees of any kind. Fivetran's pricing scales with monthly active rows; Airbyte Cloud charges per connector; dbt Cloud charges per seat. None of those pricing axes apply here.
- Single binary covering extract, transform, and loadMost open-source ETL stacks require composing separate tools: Airbyte or Meltano for EL, dbt for T, Airflow or Prefect for orchestration. Duckle covers all three in one artifact, compiled to SQL on DuckDB, with orchestration built in via schedules and plans.
- Runs entirely on infrastructure you ownPipelines execute in-process on DuckDB on hardware you provision. No telemetry, no account, and no outbound calls from Duckle itself. Documented deployment targets include Docker, EC2, ECS, EKS, Azure, and Google Cloud, and it runs on any Linux server with no desktop dependency in headless mode.
- DuckDB native speed on commodity hardwareExecution is vectorized and columnar through DuckDB. The README documents 96 million rows from Postgres to Parquet in 39.9 seconds, and multi-branch 16-node pipelines finishing in a few seconds. A bigger instance is a faster pipeline; no external warehouse is required.
Trade-offs
- -Beta status with 76 open issuesDuckle shipped its first release in May 2026 and is labeled beta. With 76 open GitHub issues and a small team at SlothFlowLabs, edge cases in the connector surface are likely. Teams with production-critical, high-reliability requirements should evaluate carefully before committing.
- -AI assistant is a 1.1 GB opt-in downloadThe Duckie AI assistant requires downloading about 1.1 GB of model weights (llama-server plus a Qwen 2.5 Coder GGUF file) separately from the core binary. On slow connections or air-gapped environments this is a barrier, though the AI feature is entirely optional for running pipelines.
- -Single-machine architecture, not a distributed systemDuckle is designed for local and small-team data work on a single machine or managed server, not to replace a distributed warehouse or cluster. Performance scales with the hardware you give it, not horizontally across nodes.
Duckle vs alternatives#
Duckle vs dbt Cloud
dbt Cloud focuses on SQL transformation for data already in a warehouse, with a managed runtime, per-seat pricing, and a cloud IDE. Duckle covers extraction, transformation, and loading in one tool, runs on infrastructure you own, and has no per-seat or per-row billing.
| Feature | Duckle | dbt Cloud |
|---|---|---|
| License | Apache-2.0 | Proprietary |
| Self-hosting | Yes (Docker, EC2, Kubernetes) | No |
| Extract and load | Yes (116 sources, 72 destinations) | No (transformation only) |
| SQL transformation | Yes (compiles to DuckDB SQL) | Yes (runs on your warehouse) |
| dbt support | Yes (runs dbt on DuckDB) | Native dbt environment |
| Warehouse required | No (DuckDB runs in-process) | Yes |
| Per-seat pricing | No | Yes |
| Visual pipeline authoring | Yes | No |
| AI assistant | Yes (on-device, no API key) | Limited |
Duckle is the better choice when you need full ETL coverage on infrastructure you own, when per-seat or per-row billing is not workable, or when your pipelines need to run without an external warehouse. Its 385 components handle the extraction and loading stages that dbt Cloud is not designed for, and it can run dbt models on DuckDB inside the same studio, giving teams that already use dbt a path to self-hosted transformation without abandoning their dbt investment.
dbt Cloud is still the right choice for teams already running dbt in production on a large warehouse like Snowflake or BigQuery, where the managed runtime, CI/CD integration, and IDE convenience justify the per-seat cost. Duckle's beta status and self-managed operations are not equivalent to dbt Cloud's managed reliability and enterprise support. Teams that only need SQL transformation on an existing warehouse, and are comfortable with the pricing, are better served by dbt Cloud's mature ecosystem.
Quick start#
Install the CLI via pip; the DuckDB engine is included automatically on Linux, macOS, and Windows.
```bash
pip install duckle
```What it's built on#
- Languages
- JavaScriptPythonRustTypeScript
- Frameworks
- React
- Infrastructure
- Kubernetes
- Tooling
- esbuild
FAQ#
Is Duckle free to use?
Yes. Duckle is Apache-2.0 licensed and free to run on any infrastructure, with no per-row, per-connector, or per-seat billing. There is no managed cloud tier requiring a paid plan; the self-hosted binary is the product.
What is the difference between Duckle and dbt Cloud?
dbt Cloud handles SQL transformation for data already inside a warehouse, with per-seat pricing and a managed runtime. Duckle covers extraction and loading in addition to transformation, runs on your own infrastructure, and has no per-seat billing. It can also run dbt models on DuckDB inside the same studio, so it can complement or replace the dbt Cloud runtime depending on where your workloads run.
Can Duckle run without a cloud or data warehouse?
Yes. Pipelines compile to SQL and execute on DuckDB, an embedded columnar engine that runs in-process. No external warehouse is required. Deploy to a server or cloud account you own and duckle-runner serve schedules them headless. It suits air-gapped and on-premises environments, and still reads from and writes to cloud systems whenever a pipeline explicitly does so.
Does Duckle send data to any third party?
No. Pipelines execute in-process on infrastructure you own. There is no telemetry, no account requirement, and no outbound calls from Duckle itself. The built-in AI assistant runs its model in-process and needs no API key, so describing a pipeline in plain English does not send your schema or data externally.
What languages can I use to author Duckle pipelines?
Pipelines can be authored on the visual canvas without writing code, in Python using the duckle SDK where expressions compile to vectorized SQL at plan time, or directly in SQL. The CLI (pip install duckle) also supports CI, cron, and container workflows with the duckle validate and duckle pipeline commands.
Similar open-source tools#
CocoIndex
Incremental data framework for AI agents.
FckSignups
Open-source tools that work instantly, no signup required
Herdr
Agent runtime that keeps terminals alive across machines
CodeBoarding
Review code changes as architecture maps, not line diffs
dbx
Manage 100+ databases with a 25 MB open source client
libredb-studio
One browser tab for 18 database engines, deployed beside your data