Transformation/Orchestration and schedulingAnalytics Engineer

How do I orchestrate dbt models without Airflow?

Run your dbt project as part of a Bruin pipeline. Bruin resolves dependencies and schedules the run, so you get orchestration for dbt models without standing up and operating Airflow.

Command

bruin run

Defined in

YAML + SQL

Works with

dbt + Bruin CLI

What you get

dbt in a pipelinedependency-aware runsscheduling without Airflow

How it works in code

/* @bruin
name: run_dbt
type: dbt
dbt:
  project_dir: ./dbt
@bruin */

Run bruin run and Bruin orchestrates the dbt run in dependency order.

About Bruin

What is Bruin?

Bruin is an open-source, AI-powered data platform. One project covers the whole path from source to answer: data ingestion (with ingestr), SQL and Python transformations, quality checks, column-level lineage, a data catalog and scheduling. On top of that data, an AI data analyst answers questions in Slack, Microsoft Teams, Google Chat, WhatsApp, Discord, Telegram, email or the browser, and builds dashboards from a prompt.

Pipelines are plain SQL, Python and YAML files in your own Git repository, and the work runs inside your own warehouse: Snowflake, BigQuery, Databricks, Redshift, Postgres, ClickHouse, DuckDB and others. The Bruin CLI is open source under Apache 2.0 and runs on a laptop, in CI or on-prem. Bruin Cloud is optional and adds scheduling, the AI data analyst, governance, SSO and audit logs.

For warehouse pipelines that means SQL and Python transformations in one project, run inside your own warehouse, with checks and tests next to the code.

Replaces
Separate tools for ingestion, transformation, orchestration, data quality, cataloguing and BI
Open source
Bruin CLI under Apache 2.0; ingestr source-available under the Functional Source License
Your data
Stays in your warehouse; pipelines stay in your Git repository
Security
ISO/IEC 27001:2022 certified and SOC 2 Type 2 attested
Used by
Data teams at Lessmore, StarBerry, NativeMinds, SCG, M8 Games and Fabrikatör

Why Bruin for transformations

  • SQL and Python together

    SQL and Python assets run in the same pipeline, share lineage and are tested with the same checks.

  • Runs in your warehouse

    Transformations run on Snowflake, BigQuery, Databricks, Redshift, Postgres, ClickHouse, DuckDB and others. Data does not leave your infrastructure.

  • Incremental by design

    Table, view, merge, delete+insert and append materializations keep runs small as tables grow.

  • Tested before it ships

    bruin validate checks the whole pipeline, and unit tests run each query against mocked inputs.

How it works

  1. 01

    Start from a template

    Run bruin init to create a project for your warehouse.

  2. 02

    Write assets

    Write SQL or Python assets with their columns, checks and materialization in the same file.

  3. 03

    Validate and run

    Run bruin validate, then bruin run, and schedule the pipeline when it works.

SQL and Python, one pipeline

Bruin CLI and ingestr are on GitHub.

A demo walks through your own data.

Sign up to our newsletter

Practical updates on open-source data pipelines, AI analysts, governance, and what we are shipping at Bruin.

The signup form is hosted by Brevo. Allow marketing cookies to load it.