Course overview/Make failure loud3 of 3

Break it on purpose

Break it on purpose

Predict, trigger, and repair controlled failures so a pipeline's checks prove their value.

Use failure as evidence

The failure drill changes one thing at a time: a duplicate key, an unexpected category, an orphan foreign key (a child ID with no matching parent), and a broken grain (the meaning of one row). Predict first, run second, and record the exact error. A query can still return plausible totals after its row meaning changes, so the drill tests the data rules, not just the syntax.

Keep each experiment isolated and restore the source before starting the next one. The check that fires latest tells you where the contract is weak. Never leave the shipped source changed.

Your task

In a disposable copy of the project, start from a clean shipped run and perform these four edits one at a time. Run only the named asset or check in dev, use bruin validate first, and restore the file before the next case:

  1. In pipeline/assets/core/dim_customer.sql, change FROM stg_customers to FROM (SELECT * FROM stg_customers UNION ALL SELECT * FROM stg_customers WHERE customer_id = 1) AS duplicated. Run bruin run --environment dev --selector fqn:dim_customer pipeline and expect dim_customer:customer_id:unique to fail.
  2. In pipeline/assets/core/fct_order_lines.sql, change COALESCE(p.category_name, 'Unknown') to COALESCE(p.category_name, 'Unexpected'). Run bruin run --environment dev --selector fqn:fct_order_lines pipeline and expect the accepted-values check to fail.
  3. First record that the clean query passes its relationship check because p.product_id is NULL for the orphan. Then change p.product_id in the first select list to i.product_id, rerun the same scoped command, and record the relationship failure for 14 orphan rows.
  4. In pipeline/assets/mart/weekly_category_revenue.sql, remove category_name from GROUP BY 1, 2 so it reads GROUP BY 1. Run bruin run --environment dev --selector fqn:weekly_category_revenue pipeline and record DuckDB's grouping error, or its exact equivalent.

Record each prediction, exact output, repair, and rerun result in docs/failure-drill.md.

Check your understanding

  • Why predict before running?
  • What does an orphan foreign key mean?
  • Why restore after each breakage?

Do it with your agent

Say next lesson, run the four isolated experiments in order, then say review my work.

Rubric

  • Records all 4 breakages in the required order with one prediction each.
  • Names the check or error that actually fired for each breakage.
  • Records the clean orphan relationship pass as a contract gap and the 14-row failure after projecting the source key.
  • Shows that each repair restored the run and identifies which breakage was caught latest.

Sign up to our newsletter

Practical updates on open-source data pipelines, AI analysts, governance, and what we are shipping at Bruin.

The signup form is hosted by Brevo. Allow marketing cookies to load it.