Bruin CLI Step 2 of 6

Shopify Data Pipeline

Guide overview
  1. Set Up Your Project & Warehouse8 min
  2. 2Ingest Your Ecommerce Data15 min
  3. 3Clean & Join Your Data15 min
  4. 4Build Ecommerce Reports15 min
  5. 5Set Up Your AI Data Analyst10 min
  6. 6Build Ecommerce Dashboards5 min
PreviousSet Up Your Project & WarehouseNextClean & Join Your Data

Create ingestion assets

Optional: install source MCPs

These platforms offer their own MCP servers that let Claude Code explore available tables and schemas before you write ingestor assets. Not required, but useful.

Official Dev MCP

npx @shopify/dev-mcp@latest

Add the Shopify connection

bruin connections add

When prompted:

  • Type: shopify
  • Name: shopify
  • API key, API secret, access token: from your Shopify app credentials
  • Shop: your store name (without .myshopify.com)

Create ingestor assets

Create a file for each table you want to ingest. These go in your ecommerce/assets/ folder.

ecommerce/assets/shopify_orders.asset.yml

name: raw.shopify_orders
type: ingestr
parameters:
  source_connection: shopify
  source_table: orders
  loader_file_format: jsonl
  incremental_strategy: merge
  incremental_key: updated_at
  primary_key: id

Tip

If you installed the Shopify MCP, ask Claude Code to list all available Shopify tables and preview the schema for each one. This helps you decide if you want to ingest additional tables like transactions or fulfillments.

Payments

Shopify Payments

If you use Shopify Payments, your payment data is already included in Shopify's order data - the financial_status, total_price, and transaction fields are all part of the orders you just ingested. No extra connection or assets needed here.

The staging layer in Step 3 will pull payment details directly from raw.shopify_orders.

Email Marketing

Klaviyo

Klaviyo covers the marketing side: how your email and SMS campaigns are performing and which customers are engaging.

Add the Klaviyo connection:

bruin connections add

When prompted:

  • Type: klaviyo
  • Name: klaviyo
  • API key: your Klaviyo private API key

Create ingestor assets:

name: raw.klaviyo_campaigns
type: ingestr
parameters:
  source_connection: klaviyo
  source_table: campaigns
  incremental_strategy: replace

Advertising

Facebook Ads

Facebook Ads shows you what you're spending on acquisition across Facebook and Instagram.

Add the connection:

bruin connections add

When prompted:

Create ingestor assets:

name: raw.facebook_campaigns
type: ingestr
parameters:
  source_connection: facebook_ads
  source_table: campaigns
  incremental_strategy: replace

Web Analytics

GA4

GA4 gives you the website funnel: who's visiting, where they come from, and what they do.

Add the connection:

bruin connections add

When prompted:

  • Type: google_analytics
  • Name: google_analytics
  • Credentials path: path to your Google service account JSON
  • Property ID: your GA4 property ID (found in GA4 Admin > Property Settings)

Create ingestor assets:

name: raw.ga4_events
type: ingestr
parameters:
  source_connection: google_analytics
  source_table: events
  incremental_strategy: merge
  incremental_key: date
  primary_key: "date,event_name"

Validate and run

Validate that all your asset definitions are correct:

bruin validate .

Or ask Claude Code:

Run bruin validate and fix any issues you find in the asset files.

Fix any errors - the most common issue is a typo in a connection name or a missing required field.

Then run the pipeline. For your first run, start with a small date range (about a month) to verify everything works before doing a full backfill:

bruin run --start-date 2025-01-01 --end-date 2025-02-01 .

This tells Bruin to only pull data within that interval. The --start-date and --end-date flags control the date range passed to each ingestor asset - see commands for all available flags.

If you have multiple environments (e.g. dev and prod), you can specify which one to use:

bruin run --start-date 2025-01-01 --end-date 2025-02-01 -e default .

Once you've confirmed the data looks right, do a full backfill with a wider range:

bruin run --start-date 2024-01-01 --end-date 2026-04-01 .

Verify data landed

Ask Claude Code to check all your tables at once:

Query each raw table in the warehouse and give me the row count for every source we ingested.

If you see row counts greater than zero, your ingestion is working. If a table is empty, ask Claude Code to troubleshoot:

The raw.shopify_orders table is empty after running the pipeline. Check the asset definition and connection, and tell me what might be wrong.

Sign up to our newsletter

Practical updates on open-source data pipelines, AI analysts, governance, and what we are shipping at Bruin.