Core Concepts
Bruin is built around a few simple but powerful concepts. This page gives a brief orientation — each concept links to its full documentation.
| Concept | Description |
|---|---|
| Pipelines | A group of assets that are executed together in dependency order |
| Assets | Anything that carries value derived from data (tables, views, files, models) |
| Variables | Dynamic values injected into your asset code during execution |
| Connections | Named configurations for authenticating with data platforms and sources |
| Commands | CLI operations to run, validate, and manage your pipelines |
| Project | A Git repository containing your pipelines, configured via .bruin.yml |
| Orchestration | How Bruin executes pipelines — scheduling, dependency resolution, concurrency, and deployment |
Orchestration
Bruin orchestrates pipeline execution through several features that work together:
- Dependencies: Assets declare their dependencies via the
dependsfield. Bruin uses this to determine execution order — assets run only after all their upstream dependencies have succeeded, and assets without dependencies on each other run in parallel automatically. - Lineage: The dependency graph forms a lineage that lets you trace how data flows through your pipeline. You can visualize it via the
lineagecommand or the VS Code lineage panel. In Bruin Cloud, lineage extends across pipelines. - Scheduling: Pipelines can be scheduled using cron expressions in
pipeline.yml, defining when and how often they run. - Concurrency: Control how many assets run simultaneously (
--workers) and how many pipeline runs can overlap (concurrencysetting in Bruin Cloud). - Deployment: Run pipelines locally, on VMs with cron, via CI/CD (GitHub Actions, GitLab), or on cloud infrastructure (AWS Lambda, ECS, Google Cloud Run).
- Bruin Cloud: Managed orchestration with scheduling, monitoring, notifications, and cross-pipeline dependencies — no infrastructure to manage.