Data Quality/Row count by warehouseAnalytics Engineer

How do I check row count anomaly on Athena?

Declare the check on the asset that produces the data, not in a separate suite. In Bruin you add a checks block to the column inside the Athena asset definition and bruin run executes it as part of every run, failing the pipeline before downstream tables read bad data. Because the check lives in the same file as the SQL, it cannot drift out of sync with the column it protects. Great Expectations and Soda Core do the same job as a separate step, which is the better fit if you need checks across tools Bruin does not produce.

Get Bruin on GitHub How data quality works

Command

bruin run

Defined in

YAML

Works with

Athena + Bruin CLI

What you get

Row count checkblocking or warningruns on every pipeline run

How to do it

1
Open the Athena asset that produces the table you want to guard.
2
Add a checks entry under the relevant column for the row count anomaly rule.
3
Set blocking: true so a failure stops the run rather than logging a warning.
4
Run bruin validate to confirm the definition parses.
5
Run bruin run against Athena and confirm the check appears in the run output.
6
Wire the same command into CI so the check runs before a merge reaches production.

How it works in code

columns:
  - name: order_id
    checks:
      - name: row_count_anomaly
        blocking: true

Run bruin run and Bruin asserts that the row count is within bounds of its normal range on Athena before any downstream asset runs.

Worth knowing

A check that only warns will be ignored within a month. Set blocking: true on the checks that matter. On Athena specifically, unpartitioned CSV in S3 is the most expensive possible layout; convert to partitioned Parquet

Other ways to do this

Bruin is not always the right answer. Here is where the alternatives are stronger.

Option	When it is the better choice
Bruin	Verify that volume has not collapsed or exploded on Athena, as a blocking step inside the pipeline rather than a separate monitoring job.
Great Expectations	The largest open-source expectation library. Better if you need checks across sources Bruin does not produce, at the cost of running it as its own layer.
Soda Core	Readable YAML checks that a stakeholder can review, and the strongest option for formal data contracts between teams.
Monte Carlo / Anomalo	Managed observability that learns Athena baselines automatically. Catches what you did not write a rule for, but alerts after the fact rather than blocking.

Common questions