ingestr recipes/Load a specific sourceData Engineer

How do I load Mixpanel data into my warehouse?

Mixpanel exposes product analytics objects such as events and user profiles through its service account API. ingestr lands these tables in your warehouse with one command. Bruin does this in one platform: ingestion, SQL and Python pipelines, quality checks, lineage, and an AI data analyst that answers in Slack, Microsoft Teams, Google Chat, WhatsApp, Discord, Telegram, email and the browser.

Command

ingestr ingest

Defined in

CLI

Works with

Mixpanel to any ingestr destination

What you get

Mixpanel API sourceno connector codeservice account auth

How it works in code

$ ingestr ingest \
  --source-uri 'mixpanel://?username=my-service-account&password=my-secret&project_id=12345' \
  --source-table 'events' \
  --dest-uri $SNOWFLAKE_URI --dest-table 'analytics.mixpanel_events'

Run ingestr ingest and ingestr loads the Mixpanel data into your warehouse.

About Bruin

What is Bruin?

Bruin is an open-source, AI-powered data platform. One project covers the whole path from source to answer: data ingestion (with ingestr), SQL and Python transformations, quality checks, column-level lineage, a data catalog and scheduling. On top of that data, an AI data analyst answers questions in Slack, Microsoft Teams, Google Chat, WhatsApp, Discord, Telegram, email or the browser, and builds dashboards from a prompt.

Pipelines are plain SQL, Python and YAML files in your own Git repository, and the work runs inside your own warehouse: Snowflake, BigQuery, Databricks, Redshift, Postgres, ClickHouse, DuckDB and others. The Bruin CLI is open source under Apache 2.0 and runs on a laptop, in CI or on-prem. Bruin Cloud is optional and adds scheduling, the AI data analyst, governance, SSO and audit logs.

For data ingestion that means ingestr: one command copies data from a source to a destination, and the same project transforms and checks it afterwards.

Replaces
Separate tools for ingestion, transformation, orchestration, data quality, cataloguing and BI
Open source
Bruin CLI under Apache 2.0; ingestr source-available under the Functional Source License
Your data
Stays in your warehouse; pipelines stay in your Git repository
Security
ISO/IEC 27001:2022 certified and SOC 2 Type 2 attested
Used by
Data teams at Lessmore, StarBerry, NativeMinds, SCG, M8 Games and Fabrikatör

Why Bruin for data ingestion

  • One command per source

    ingestr copies a table or API from a source to a destination with one command, or one small YAML asset in a Bruin project.

  • Fast

    In Bruin’s published benchmarks against Sling, dlt, Airbyte and Spark, ingestr was the fastest in 76 of 79 source, destination and size combinations.

  • Incremental and CDC

    Load only new rows with an incremental key, merge on a primary key, or stream changes from databases with change data capture.

  • Part of the pipeline

    The load is an asset in the same project as the SQL and Python that use it, so lineage and checks cover it too.

How it works

  1. 01

    Define the source and destination

    Write an ingestr asset with the source connection, the source table and the destination.

  2. 02

    Choose a strategy

    Pick replace, append, merge or delete+insert, and an incremental key if the source has one.

  3. 03

    Run and schedule

    Run it with bruin run, add checks on the loaded columns, and schedule it in Bruin Cloud or your own scheduler.

Move data with one command

Bruin CLI and ingestr are on GitHub.

A demo walks through your own data.

Sign up to our newsletter

Practical updates on open-source data pipelines, AI analysts, governance, and what we are shipping at Bruin.

The signup form is hosted by Brevo. Allow marketing cookies to load it.