Bruin CLI Step 6 of 6

ClickHouse + Bruin 101

1) Run the deterministic ClickHouse core

Run from the parent directory. The PostgreSQL assets are tagged requires-postgres-default, so exclude them while using only the local ClickHouse service:

bruin run clickhouse-bruin-101/pipeline.yml \
  --environment default \
  --exclude-tag requires-postgres-default \
  --full-refresh \
  --start-date 2024-04-01 \
  --end-date 2024-04-15

--full-refresh rebuilds destination tables. It is appropriate for this disposable local walkthrough, but do not use it against production by habit. The date window drives the time_interval, append, and operational snapshot examples.

Read the command from top to bottom:

  • bruin run clickhouse-bruin-101/pipeline.yml starts from the pipeline file, so Bruin can build the full asset graph.
  • --environment default selects the connections under environments.default in .bruin.yml.
  • --exclude-tag requires-postgres-default leaves out assets that need the optional PostgreSQL connection.
  • --full-refresh tells materialized tables to rebuild for this tutorial run.
  • --start-date and --end-date render the Jinja date variables used by the incremental SQL assets.

When both connections are configured, remove --exclude-tag requires-postgres-default to run the whole graph. The ClickHouse template reference remains the source of truth if the template changes.

Sign up to our newsletter

Practical updates on open-source data pipelines, AI analysts, governance, and what we are shipping at Bruin.