← All changes ·

2026 W21

Variant pipelines, BigQuery cost limits, DuckDB lakehouse on S3-compatible storage, and more.

I am super excited to have variant pipelines out in this release. Variant pipelines allow you to have one pipeline definition that generates multiple pipelines based on different variable values. This is a game-changer for managing similar pipelines with small differences, like dev/prod or different regions.

name: "{{ var.client }}_pipe"
schedule: "{{ var.schedule }}"

variables:
  client:
    type: string
    default: client_a
  region:
    type: string
    default: us
  schedule:
    type: string
    default: "@daily"

variants:
  client_alpha:
    client: alpha
    region: us
    schedule: "@hourly"
  client_beta:
    client: beta
    region: eu
    schedule: "0 6 * * *"

Bruin CLI

New Features

  • Variant pipelines: Pipelines can now declare a variants: block in pipeline.yml that binds values to variables:, materializing one template pipeline into multiple concrete pipelines. Variants are selectable from run, validate, render, lineage, and patch.
  • BigQuery cost guards: New max_billable_bytes, max_query_cost, max_billable_bytes_soft, and max_query_cost_soft connection settings dry-run each BigQuery query and block (or warn) if estimates exceed the limit. The bruin query command gains --dangerously-bypass-soft-limits to override soft limits.
  • Query execution summaries: bruin query now reports execution metadata (rows affected, bytes processed) for DDL/DML statements that return no rows on BigQuery, Postgres, and DuckDB. JSON output gains an execution field.
  • BigQuery job IDs printed in output: Run and query commands now log the BigQuery job ID for main tasks, column/custom checks, and sensors, so you can jump straight to the BigQuery console.
  • Wider materialization support:
    • The ddl strategy (table created from a column definition rather than a query) is now supported on Fabric, Oracle, Synapse, MSSQL, Trino, and Vertica.
    • Oracle now supports create+replace, append, delete+insert, truncate+insert, merge, time_interval, SCD2_by_time, and view materialization, plus Oracle column checks and metadata push.
  • Partial run results on abort: bruin run exits cleanly on Ctrl+C, prints an "aborted" summary, and reports which assets succeeded, failed, were skipped, or never started.
  • Pipeline-level full-refresh protection: Set full_refresh_restricted: true at the asset level or under config.full_refresh_restricted in .bruin.yml to protect specific tables from being dropped during --full-refresh.
  • Configurable sensor timeout: Sensor assets accept a timeout parameter (e.g. timeout: 2h) that overrides the default 24 hour cap. Snowflake's sf.sensor.query honors this timeout as well.
  • Repeatable --exclude-tag: bruin run --exclude-tag (-x) now accepts multiple values, and missing exclude tags warn instead of aborting the run.
  • --split-rows for CSV export: bruin query --export now accepts --split-rows N to break large CSV exports into multiple parts, each named query_result_<timestamp>_partN.csv.
  • Query annotations: A new --query-annotations flag (and BRUIN_QUERY_ANNOTATIONS env var) tags ad-hoc queries with arbitrary metadata for warehouse query history. Ad-hoc queries now always carry a baseline @bruin.config: {"type":"adhoc_query"} annotation.
  • "Still running" progress logs: For tasks running longer than 120 seconds, Bruin prints a Still running: <id> (<elapsed>) line every 10 seconds so long-running work no longer looks stuck.
  • Python materialize() is more forgiving: materialize() can return None or yield zero items to skip materialization with a warning instead of crashing. Generators can yield individual dicts or batches.
  • DuckDB lakehouse on S3-compatible storage: storage.s3 accepts endpoint, url_style, and use_ssl, unlocking MinIO, Cloudflare R2, Backblaze B2, Tigris, and on-prem S3.
  • DuckDB complex column support: DuckDB queries now return native types for LIST, STRUCT, and MAP columns so JSON output serializes cleanly.
  • Ingestion engine pinning: A new parameters.version field on ingestr assets selects the ingestion engine and version (v0 for the Python ingestr, v1+ for the Go binary, with optional minor/patch pins).
  • Bruin Cloud client retries on 429: bruin cloud ... commands retry on HTTP 429 with Retry-After backoff instead of failing on the first throttle. bruin cloud diagnose no longer requires the project to be passed explicitly.
  • New project templates: A bruin-cloud template ingests the Bruin Cloud API into DuckDB, and a new bigquery template ships with macros and seeds.

Bug Fixes

  • Fixed a scheduler deadlock on pipelines with more than 100 initially eligible tasks when running with -t <tag>.
  • Fixed BigQuery hook-wrapped SQL failing with "DECLARE must be at the start of a script" when pre-hooks contain SET/IF/BEGIN; auto-generated DECLAREs are now hoisted to the top.
  • Fixed bruin query --dry-run --output csv failing instead of falling back to plain output.
  • Fixed DuckDB SCD2 timestamp metadata type mismatches.
  • Notification config (Slack, Microsoft Teams, Discord) is now validated uniformly at pipeline, asset, and check level, with check-level context in error messages.

Bruin Cloud

Onboarding and Sign-up

  • Progressive onboarding: The previous 6-step setup wizard is replaced with a simple welcome page (organization name only) and dashboard widgets that guide users through the rest of setup over time.
  • Continue with Google: Login and register pages now support signing in or registering with a Google account.
  • "I already have an organization" flow: Users waiting on a team invite can skip the create-organization step; the waiting page polls and auto-redirects once they're added.
  • Unified Getting Started page: AI data analyst setup and pipeline setup are combined into one Getting Started page with a side-by-side layout and an embedded docs/video side panel.
  • All users can create new teams, even when they already own one.

AI Chat and Agents

  • Pin and rename threads: AI chat threads can be pinned to the top and renamed inline via a hover menu. Dashboard chat tabs gain a three-dots menu with Rename and Delete entries.
  • Inline dashboard rename: Click the dashboard title in edit mode to rename it without leaving the canvas.
  • Connection Sets editor: Connection Sets are now expandable rows showing connection counts and types at a glance, with autosave on add/remove. A "Create new Connection Set" link is exposed from the agent edit modal.
  • Agents without a project: Agents can be created without a project and run with an empty workspace.
  • File uploads in scheduled runs: Scheduled runs accept file uploads with drag-and-drop, document icons, and a removable file list. Scheduled-agent autosave is replaced by an explicit Save button to avoid dropping pasted content.
  • "No rows returned" indicator in the AI chat SQL panel when a query returns zero rows.
  • Auto-scroll agent thinking panel so new tool calls stay in view during execution.
  • Live progress in Slack: Slack agent replies now show a native "is..." status pill that rotates through phase-aware messages, plus an in-thread progress message that updates with the actual commands the agent is running and evolves into the final response. Non-SQL replies now include the thread link too.
  • Microsoft Teams welcome message when the bot is added to a conversation.
  • Google Chat welcome card when the bot is added to a space, with refined help command responses.
  • WhatsApp inactivity reset: Expired WhatsApp sessions fall back to templates, and a "Continue" button resends pending scheduled-run results.

Pipelines, Assets, and Runs

  • Rendered SQL toggle on asset detail: A Source/Rendered switch in the asset Code card shows SQL after Jinja interpolation.
  • Interval modifiers indicator: A clock icon and tooltip surface interval modifiers next to asset names in pipeline tables, the asset index, and lineage views.
  • Show in lineage button: Each asset row on the run detail page has a button that opens the lineage view zoomed and centered on that asset, with upstream/downstream dependencies highlighted and unrelated nodes faded.
  • Historical runs show removed assets: Old runs now load the pipeline definition at the run's version, so removed assets still appear (faded, with "removed" badges and a warning banner).
  • Run selected assets: When assets are selected on the pipeline page, the top "+ New Run" button triggers a selective run with the label reflecting the count.
  • Connection filter on the assets overview sidebar; asset list is now sortable by connection.
  • Asset-type badges in brand colors: Lineage nodes and asset-type badges use recognizable per-platform brand colors (BigQuery, Snowflake, dbt, etc.).
  • Commit SHAs link to GitHub commit pages.
  • Enable Anyway button on the enable-pipeline dialog to bypass missing connections.

Dashboards

  • Date tooltip headers: ISO date x-axis values now render as locale-formatted dates in chart tooltip headers (line, area, bar, heatmap, histogram), including YYYY-MM monthly buckets like "Nov 2025".
  • Tooltips can render outside widget bounds, so tooltips at the top of a chart are no longer clipped.
  • Table widgets use scroll instead of pagination in AI Dashboard widgets.
  • YAML view of dashboard state in the side panel.
  • Project names displayed in the dashboard pipeline list.

Glossary

  • Optimistic updates: Glossary edits and new entities appear immediately while GitHub propagation catches up.

Billing and Usage

  • Free credit visibility: AI Usage adds Messages and Threads per Month charts with limit lines, plus a $100 free credit and 10h memory-hour KPI on the Usage Dashboard. The Free Credit card shows an exhausted state when fully consumed.
  • Date range presets: Yesterday and Last month are now options on pay-as-you-go usage, and Cost Explorer date filters are now date-only with explicit Apply and reset behavior.
  • Card-add settles outstanding balance instead of getting stuck after a hard limit.
  • Disabled run/enable UI for billing-blocked pay-as-you-go teams to prevent dead clicks.

Bug Fixes

  • Fixed stuck Slack placeholder messages when answers exceed the 4000 character chat.update limit (now falls back to chat.postMessage).
  • Fixed stale check results from prior attempts marking passing assets as failed on retries.
  • Fixed activity tab instance links returning 404 for bulk reruns.
  • Fixed duplicate WhatsApp session_followup templates when sessions expired.
  • Fixed custom-cron validation in scheduled agents causing every custom cron save to fail.
  • Fixed team deletion leaving users with an invalid current_team_id.
  • Fixed chart hover flicker caused by a scrollbar feedback loop.
  • Fixed paygo subscriptions stuck when cancelled and the card was re-added.
  • Fixed double-counting of coupon usage and cached Stripe errors.
  • Fixed the dashboard parsing banner staying visible permanently.
  • Fixed null GitHub installation tokens killing sandbox jobs when an app was revoked.
  • Fixed project password changes not propagating to project tokens, leaving agents with stale credentials.

Bruin VS Code

  • Variant selector in the asset panel: For pipelines that declare variants in their pipeline config, a Variant dropdown appears next to the environment selector. The selected variant is appended to the run command as --variant "<name>". The dropdown defaults to the first variant alphabetically and is hidden for pipelines without variants.
  • Edit/View toggle for Ingestr assets: The Ingestr asset panel now defaults to a read-only rendered view and exposes Edit / Done links in the section header to switch to the editable parameter form. Variables in YAML parameters are resolved server-side via bruin render so you see real values instead of templates.
  • Bug fix: Panel now refreshes on file save on Windows. Recent VS Code versions on Windows returned paths in a format that did not match what Bruin CLI returned, so save events were dropped.