Guide
Build a local DuckLake lakehouse
Build a local DuckLake lakehouse with Bruin, DuckDB, Postgres, and MinIO - then run a pipeline that seeds, ingests, and aggregates data.
What
Build a local DuckLake lakehouse with DuckDB as the query engine, Postgres as the DuckLake catalog, and MinIO as S3-compatible storage. The finished Bruin pipeline seeds a CSV into DuckDB, ingests it into DuckLake, then creates an aggregate table in the lakehouse.
DuckLake support is currently in beta. See the DuckLake configuration reference for supported catalog and storage combinations.
How
First, run Postgres and MinIO with Docker. Then configure a local DuckDB connection for the source data and a DuckLake connection that points to Postgres and MinIO. Finally, use a seed asset, an Ingestr asset, and a DuckDB SQL asset to build the pipeline:
CSV -> DuckDB -> DuckLake -> DuckLake aggregate
source Postgres + MinIO
Before you start
- Bruin CLI installed
- Docker Desktop or OrbStack installed and running
- Familiarity with Bruin Core Concepts (recommended)
Guide overview
- 1Create the projectCreate a Git repository and initialize an empty Bruin pipeline for the DuckLake lakehouse.
- 2Start Postgres and MinIORun a local Postgres catalog, MinIO object store, and DuckLake bucket with Docker Compose.
- 3Configure the DuckLake connectionConfigure the local DuckDB source, the DuckLake catalog and storage, and the pipeline defaults.
- 4Build, run, and inspect the pipelineSeed a local CSV, ingest it into DuckLake, aggregate it, and inspect the resulting lakehouse tables.
Get help & contribute