Anthropic + Bruin

SourceDestination

Ingest data from Anthropic or push enriched data back — with quality checks, lineage, and scheduling. Defined in YAML, version-controlled in Git.

For business teams

What you get

API data, on schedule
Anthropic data lands in your warehouse automatically. No scripts to maintain, no pagination to handle.
Only fetch what changed
Incremental sync means no re-processing. Bruin tracks watermarks so you only get new and updated records.
Catch API changes early
Quality checks validate response data on every sync. Schema changes or missing fields get caught before they break models.
Transform in the same pipeline
Reshape Anthropic API data with SQL or Python. Compute metrics, normalize schemas, and build models — all version-controlled.

For data & engineering teams

How it works

Managed pagination & retries
Bruin handles Anthropic API pagination, rate limiting, and retries. You define the source — Bruin does the rest.
YAML-defined, Git-versioned
Your Anthropic pipeline is a YAML file. Review in PRs, deploy with CI/CD, roll back with git revert.
Incremental with watermarks
Bruin tracks cursor positions and watermarks. Only new and updated Anthropic records get fetched on each run.
Schema validation on responses
Quality checks validate Anthropic API response structure on every sync. Catch breaking API changes early.

Before you start

Admin API key (organization admin role required)

Organization setup in Console → Settings → Organization

Step 1

Add your Anthropic connection

Connect using Admin API key (organization admin required). Add this to your Bruin environment file — credentials are stored securely and referenced by name in your pipeline YAML.

Parameters

api_keyAnthropic Admin API key (must start with sk-ant-admin...)

connections:
  anthropic:
    type: anthropic
    uri: "anthropic://?api_key=<admin_api_key>"

Step 2

Create your pipeline

Define a YAML asset that tells Bruin what to pull from Anthropic and where to land it. This file lives in your Git repo — reviewable, version-controlled, and deployable with CI/CD.

Available tables

claude_code_usageusage_reportcost_reportorganizationworkspacesapi_keysinvitesusersworkspace_members

name: raw.anthropic_claude_code_usage
type: ingestr

parameters:
  source_connection: anthropic
  source_table: 'claude_code_usage'
  destination: bigquery

Step 3

Add quality checks

Add column-level and custom SQL checks to your Anthropic data. If a check fails, the pipeline stops — bad data never reaches downstream models or dashboards.

Validate API data freshness on every sync

Ensure record IDs are unique across fetches

Catch missing fields from API response changes

columns:
  - name: id
    checks:
      - name: not_null
      - name: unique
  - name: fetched_at
    checks:
      - name: not_null

custom_checks:
  - name: API data is fresh
    query: |
      SELECT MAX(fetched_at) >
        TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 24 HOUR)
      FROM raw.anthropic_claude_code_usage

Step 4

Run it

One command. Bruin connects to Anthropic, pulls data incrementally, runs your quality checks, and lands clean data in your warehouse. If a check fails, the pipeline stops — bad data never reaches downstream.

Backfill historical data with --start-date

Schedule with cron or trigger from CI/CD

Full lineage from Anthropic to your dashboards

$ bruin run .

Running pipeline...

  anthropic_claude_code_usage
    ✓ Fetched 2,847 new records
    ✓ Quality: campaign_id not_null     PASSED
    ✓ Quality: spend not_null           PASSED
    ✓ Quality: no negative ad spend     PASSED
    ✓ Loaded into bigquery

  Completed in 12s

Other API integrations

Allium

Frankfurter

HTTP

ISOC Pulse

Socrata

Ready to connect Anthropic?

Start for free, or book a demo to see how Bruin handles ingestion, quality, lineage, and scheduling for your entire data stack.

Book a demo Read the docs