Variables

Video

Built-in variables

Every pipeline run automatically provides start date and end date variables based on the pipeline schedule. For a monthly pipeline, the start date is the beginning of the month and the end date is the end of the month.

In SQL assets

Built-in variables are injected using Jinja syntax:

WHERE pickup_datetime >= '{{ start_date }}'
  AND pickup_datetime < '{{ end_date }}'

You can preview the compiled query in the Bruin render panel to see the actual dates injected.

In Python assets

Built-in variables are provided as environment variables:

import os

start_date = os.environ["BRUIN_START_DATE"]
end_date = os.environ["BRUIN_END_DATE"]

Custom variables

You define custom variables in your pipeline.yml:

variables:
  - name: taxi_types
    type: array
    default:
      - yellow

Custom variables are accessed in Python assets via environment variables under the BRUIN_VARS_ prefix. In SQL assets, they are available as Jinja variables.

Overriding variables at runtime

You can override any custom variable when creating a run - either from the VS Code extension panel or from the CLI:

bruin run --var taxi_types='["yellow","green"]' pipeline.yml

This is useful for ad-hoc runs where you want different parameters without changing the pipeline configuration.

Key points

• Start and end dates are always available and derived from the pipeline schedule
• SQL uses Jinja ({{ start_date }}), Python uses environment variables (os.environ)
• Custom variables are defined in pipeline.yml with defaults
• Any variable can be overridden at runtime for ad-hoc runs