Build a full pipeline next

Course recap

• Define a model's grain, primary key, and owner before you write its transformation.
• Keep source-specific cleanup in staging models, then use analytics models to answer a clear business question.
• Choose a materialization and incremental strategy that matches how the source changes.
• Add checks, validate a narrow interval, inspect the output, and review the result with the person who uses it.
• Use Python and agents where they help, but keep the pipeline, context, and verification steps visible.

First project tips

Choose a question that has a real user and a small source. For example: "What revenue did each customer generate each day?" or "Which subscriptions changed status this week?"

The first version does not need every source or every metric. It needs a clear grain, a working run, and enough checks to show whether the result is usable.

Glossary

• Source is the system or raw table that provides the data. Start with one source so you can understand its keys, update pattern, and timestamps.
• Staging model cleans and standardizes the source. It gives later models a stable input and keeps source-specific logic in one place.
• Analytics model is the dataset built for a business question. It should have a documented grain, key, and owner.
• Check tests an assumption about the model or source. Start with checks for duplicate keys, missing required values, and the freshness or completeness that matters to the question.
• Run interval is the date or timestamp range used to test and schedule the model. A narrow interval makes it easier to compare the output with source records.
• Owner is the person or team responsible for deciding whether the model definition and output are correct. Ownership keeps a metric from becoming an unnamed shared assumption.

First project shape

source data
  -> staging model
  -> analytics model
  -> checks and a scheduled run

This is enough for a first project. Add a second source, a more complex join, or a downstream report only when the first path is working and reviewed.

Build order

1. Write the question, grain, and primary key in the asset metadata or pull request.
2. Connect or ingest one source in a development environment.
3. Build a staging model and inspect its rows.
4. Build the analytics model and add the checks that protect its grain.
5. Validate, render, and run a narrow interval.
6. Review the result with the person who needs the data, then document the assumptions.

Add Python or an agent when it fits the work

• Use Python when the transformation needs a library or logic that would be awkward in SQL. Keep the asset in the same pipeline so dependencies and runs stay visible.
• Use an agent to inspect project context, explain an unfamiliar asset, or draft a change. Give it a limited scope and review the generated work before running it.

Official links

• Bruin templates guide
• ingestr supported sources
• Bruin documentation
• Bruin MCP documentation