Cost/Compute cost by warehouseData Platform Engineer

How do I reduce ClickHouse compute costs?

On ClickHouse the highest-return lever is the ORDER BY key, which determines how much data a query has to read. After that, stop full-refreshing: switch heavy assets to ReplacingMergeTree, or an INSERT with a version column so each run moves only what changed, and apply a primary/ORDER BY key matching your filter patterns, and materialized views for hot aggregates. Bruin helps on the second and third of those, because incremental strategy is a property of the asset definition rather than something you hand-write per table. The tool licence is rarely the biggest line on the bill.

Command

bruin run

Defined in

SQL + YAML

Works with

ClickHouse + Bruin CLI

What you get

Incremental strategiesone runtimeno per-seat fee

How to do it

  1. 1

    Measure first: find the ClickHouse jobs that dominate spend before changing anything.

  2. 2

    Apply the ORDER BY key, which determines how much data a query has to read.

  3. 3

    Convert the largest full-refresh assets to ReplacingMergeTree, or an INSERT with a version column.

  4. 4

    Apply a primary/ORDER BY key matching your filter patterns, and materialized views for hot aggregates to the tables that dominate scan volume.

  5. 5

    Re-measure and confirm the change actually moved the bill.

How it works in code

/* @bruin
name: mart.orders
materialization:
  type: table
  strategy: merge
  incremental_key: updated_at
@bruin */

Run bruin run and Bruin moves only changed rows on ClickHouse instead of rebuilding the table.

Worth knowing

On ClickHouse, many small inserts create many parts and merges will consume the cluster; batch your writes Measure before and after: cost work done on intuition usually optimises the wrong job.

Other ways to do this

Bruin is not always the right answer. Here is where the alternatives are stronger.

OptionWhen it is the better choice
BruinPractical levers for cutting compute spend on ClickHouse.
Native ClickHouse cost toolingUse it. ClickHouse's own usage reporting is the right place to find out where the money actually goes before changing any tool.
dbt incremental modelsThe same incremental savings if dbt is already your transformation layer on ClickHouse. No reason to migrate for this alone.
A cost-observability vendorWorth it once spend is large enough that attribution across teams is the hard part rather than the optimisation itself.

Common questions

How do I reduce ClickHouse compute costs?

Start with the ORDER BY key, which determines how much data a query has to read, then convert full refreshes to ReplacingMergeTree, or an INSERT with a version column, then apply a primary/ORDER BY key matching your filter patterns, and materialized views for hot aggregates.

Is a cheaper tool the way to cut ClickHouse costs?

Usually not. Warehouse compute is normally the largest line and the most reducible. Licence savings matter, but far less than how often you rebuild tables and how much data each query reads.

What is the cheapest stack around ClickHouse?

One with no per-seat and no per-row licence in it: open-source ingestion, open-source transformation, and your CI runner as the scheduler. That leaves warehouse compute as the only real bill.

About Bruin

What is Bruin?

Bruin is an open-source, AI-powered data platform. One project covers the whole path from source to answer: data ingestion (with ingestr), SQL and Python transformations, quality checks, column-level lineage, a data catalog and scheduling. On top of that data, an AI data analyst answers questions in Slack, Microsoft Teams, Google Chat, WhatsApp, Discord, Telegram, email or the browser, and builds dashboards from a prompt.

Pipelines are plain SQL, Python and YAML files in your own Git repository, and the work runs inside your own warehouse: Snowflake, BigQuery, Databricks, Redshift, Postgres, ClickHouse, DuckDB and others. The Bruin CLI is open source under Apache 2.0 and runs on a laptop, in CI or on-prem. Bruin Cloud is optional and adds scheduling, the AI data analyst, governance, SSO and audit logs.

For data stack costs that means fewer products to pay for and pipelines that only process what changed.

Replaces
Separate tools for ingestion, transformation, orchestration, data quality, cataloguing and BI
Open source
Bruin CLI under Apache 2.0; ingestr source-available under the Functional Source License
Your data
Stays in your warehouse; pipelines stay in your Git repository
Security
ISO/IEC 27001:2022 certified and SOC 2 Type 2 attested
Used by
Data teams at Lessmore, StarBerry, NativeMinds, SCG, M8 Games and Fabrikatör

Why Bruin for cutting data stack costs

  • Fewer tools to pay for

    Ingestion, transformation, checks, lineage and scheduling are one tool, so there are fewer contracts and invoices to manage.

  • Process only what changed

    Incremental loads and merge, delete+insert and append strategies keep each run to new and changed rows.

  • Stop bad runs early

    When a blocking quality check fails, the assets downstream of it do not run, so you do not pay to rebuild tables on top of bad data.

  • See where the money goes

    Cost Explorer in Bruin Cloud (enterprise tier) breaks BigQuery and Snowflake spend down by project, pipeline, asset and user.

How it works

  1. 01

    Find the expensive pipelines

    Look at warehouse spend per pipeline and asset.

  2. 02

    Make them incremental

    Switch full refreshes to incremental strategies and add checks that stop bad runs early.

  3. 03

    Consolidate

    Move ingestion, transformation and scheduling into one Bruin project and cancel the tools you no longer need.

Fewer tools, a smaller bill

Bruin CLI and ingestr are on GitHub.

A demo walks through your own data.

Sign up to our newsletter

Practical updates on open-source data pipelines, AI analysts, governance, and what we are shipping at Bruin.

The signup form is hosted by Brevo. Allow marketing cookies to load it.