Allium + Bruin

Source

Ingest Allium data into your warehouse with incremental loading, quality checks, and full lineage. Defined in YAML, version-controlled in Git.

For business teams

What you get

API data, on schedule
Allium data lands in your warehouse automatically. No scripts to maintain, no pagination to handle.
Only fetch what changed
Incremental sync means no re-processing. Bruin tracks watermarks so you only get new and updated records.
Catch API changes early
Quality checks validate response data on every sync. Schema changes or missing fields get caught before they break models.
Transform in the same pipeline
Reshape Allium API data with SQL or Python. Compute metrics, normalize schemas, and build models — all version-controlled.

For data & engineering teams

How it works

Managed pagination & retries
Bruin handles Allium API pagination, rate limiting, and retries. You define the source — Bruin does the rest.
YAML-defined, Git-versioned
Your Allium pipeline is a YAML file. Review in PRs, deploy with CI/CD, roll back with git revert.
Incremental with watermarks
Bruin tracks cursor positions and watermarks. Only new and updated Allium records get fetched on each run.
Schema validation on responses
Quality checks validate Allium API response structure on every sync. Catch breaking API changes early.

Before you start

Allium account and API key

Query ID from Allium explorer

Step 1

Add your Allium connection

Connect using API key authentication. Add this to your Bruin environment file — credentials are stored securely and referenced by name in your pipeline YAML.

Parameters

api_keyThe API key for authentication with Allium API

connections:
  allium:
    type: allium
    uri: "allium://?api_key=<api_key>"

Step 2

Create your pipeline

Define a YAML asset that tells Bruin what to pull from Allium and where to land it. This file lives in your Git repo — reviewable, version-controlled, and deployable with CI/CD.

Available tables

query:<query_id>

name: raw.allium_query:<query_id>
type: ingestr

parameters:
  source_connection: allium
  source_table: 'query:<query_id>'
  destination: bigquery

Step 3

Add quality checks

Add column-level and custom SQL checks to your Allium data. If a check fails, the pipeline stops — bad data never reaches downstream models or dashboards.

Validate API data freshness on every sync

Ensure record IDs are unique across fetches

Catch missing fields from API response changes

columns:
  - name: id
    checks:
      - name: not_null
      - name: unique
  - name: fetched_at
    checks:
      - name: not_null

custom_checks:
  - name: API data is fresh
    query: |
      SELECT MAX(fetched_at) >
        TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 24 HOUR)
      FROM raw.allium_query:<query_id>

Step 4

Run it

One command. Bruin connects to Allium, pulls data incrementally, runs your quality checks, and lands clean data in your warehouse. If a check fails, the pipeline stops — bad data never reaches downstream.

Backfill historical data with --start-date

Schedule with cron or trigger from CI/CD

Full lineage from Allium to your dashboards

$ bruin run .

Running pipeline...

  allium_query:<query_id>
    ✓ Fetched 2,847 new records
    ✓ Quality: campaign_id not_null     PASSED
    ✓ Quality: spend not_null           PASSED
    ✓ Quality: no negative ad spend     PASSED
    ✓ Loaded into bigquery

  Completed in 12s

Other API integrations

Anthropic

Frankfurter

HTTP

ISOC Pulse

Socrata

Ready to connect Allium?

Start for free, or book a demo to see how Bruin handles ingestion, quality, lineage, and scheduling for your entire data stack.

Book a demo Read the docs