Pipelines in CI/Pipelines in CI/CDData Platform Engineer

How do I roll back a pipeline change that broke production?

Because the pipeline is code, git revert plus a redeploy restores the previous known-good version, with the change logged in history. Bruin does this in one platform: ingestion, SQL and Python pipelines, quality checks, lineage, and an AI data analyst that answers in Slack, Microsoft Teams, Google Chat, WhatsApp, Discord, Telegram, email and the browser.

Command

bruin run

Defined in

YAML + CI

Works with

git + Bruin CLI

What you get

CI validationgated deploysgit rollback

How it works in code

# .github/workflows/data.yml
- run: bruin validate ./pipeline
- run: bruin run ./pipeline

Run bruin run and your data pipelines follow the same review and deploy path as your app.

About Bruin

What is Bruin?

Bruin is an open-source, AI-powered data platform. One project covers the whole path from source to answer: data ingestion (with ingestr), SQL and Python transformations, quality checks, column-level lineage, a data catalog and scheduling. On top of that data, an AI data analyst answers questions in Slack, Microsoft Teams, Google Chat, WhatsApp, Discord, Telegram, email or the browser, and builds dashboards from a prompt.

Pipelines are plain SQL, Python and YAML files in your own Git repository, and the work runs inside your own warehouse: Snowflake, BigQuery, Databricks, Redshift, Postgres, ClickHouse, DuckDB and others. The Bruin CLI is open source under Apache 2.0 and runs on a laptop, in CI or on-prem. Bruin Cloud is optional and adds scheduling, the AI data analyst, governance, SSO and audit logs.

For CI/CD that means a pipeline made of files in Git that can be validated, unit-tested and deployed like any other code.

Replaces
Separate tools for ingestion, transformation, orchestration, data quality, cataloguing and BI
Open source
Bruin CLI under Apache 2.0; ingestr source-available under the Functional Source License
Your data
Stays in your warehouse; pipelines stay in your Git repository
Security
ISO/IEC 27001:2022 certified and SOC 2 Type 2 attested
Used by
Data teams at Lessmore, StarBerry, NativeMinds, SCG, M8 Games and Fabrikatör

Why Bruin for pipelines in CI/CD

  • Everything is a file

    Assets, checks, schedules and connections are files in Git, so every change is a reviewable diff.

  • Validation before anything runs

    bruin validate checks the whole pipeline and dry-runs queries on BigQuery and Snowflake.

  • Unit tests without production data

    bruin unit-test runs each query against mocked input rows and compares the output, writing nothing to the warehouse.

  • Works with your CI

    A setup-bruin GitHub Action, with documented setups for GitLab CI, Azure Pipelines, CircleCI and Jenkins.

How it works

  1. 01

    Install in CI

    Add the setup-bruin action, or install the CLI with one command in any runner.

  2. 02

    Test on every pull request

    Run bruin validate and bruin unit-test, and run quality checks against a dev environment.

  3. 03

    Deploy on merge

    Merge to main and let Bruin Cloud or your scheduler pick up the new version.

Deploy data pipelines like software

Bruin CLI and ingestr are on GitHub.

A demo walks through your own data.

Sign up to our newsletter

Practical updates on open-source data pipelines, AI analysts, governance, and what we are shipping at Bruin.

The signup form is hosted by Brevo. Allow marketing cookies to load it.