The Best Way to Replicate a Database into BigQuery (2026)

TL;DR: The best way to replicate a database into BigQuery depends on your constraints. For open-source and code-first with no server, use ingestr, Bruin's open-source ingestion CLI, which moves Postgres, MySQL, SQL Server, and more into BigQuery with one command and native load jobs. For Google-native log-based CDC, use Datastream. For fully managed across many sources, use Fivetran. Most analytics teams want a scheduled incremental load, not streaming, and ingestr covers that in one command. Use Bruin if you want that load scheduled alongside the models and tests that read from it.

Replicating a database into BigQuery means keeping BigQuery tables in sync with your source so analytics and models read from BigQuery instead of your production database. What matters: loading through BigQuery's efficient load jobs (not streaming inserts you do not need), syncing only what changed, partitioning for cost, and avoiding BigQuery's specific billing traps. Here is the practical path.

Your options

If you are all-in on Google Cloud and want real-time CDC, Datastream is the native answer. If you want open-source, code-first, and no infrastructure, ingestr, Bruin's open-source ingestion CLI, is the shortest path. Use Bruin itself if you also want the models and tests downstream of the replication in the same project.

Replicate into BigQuery with ingestr, step by step

ingestr, Bruin's open-source ingestion CLI,. Give it a source URI and a BigQuery destination URI and it loads through BigQuery's batch load jobs, which are free to run and far more efficient than streaming inserts for replication.

1. Install it.

pip install ingestr

2. Full load from Postgres into BigQuery (auth via a service-account key):

ingestr ingest \
  --source-uri 'postgresql://user:pass@host:5432/appdb' \
  --source-table 'public.orders' \
  --dest-uri 'bigquery://my-project?credentials_path=/path/to/key.json&location=EU' \
  --dest-table 'raw.orders'

3. Incremental so subsequent runs move only changed rows:

ingestr ingest \
  --source-uri 'postgresql://user:pass@host:5432/appdb' \
  --source-table 'public.orders' \
  --dest-uri 'bigquery://my-project?credentials_path=/path/to/key.json&location=EU' \
  --dest-table 'raw.orders' \
  --incremental-strategy merge \
  --incremental-key updated_at \
  --primary-key id

Schedule that command and you have continuous replication into BigQuery with no server. Swap the source URI for MySQL, SQL Server, or Oracle as needed.

Incremental vs CDC for BigQuery

• Incremental (query-based) is simplest and fits most analytics workloads. It syncs at your interval on an updated_at or incrementing id. It does not catch hard deletes unless you soft-delete.
• Log-based CDC via Datastream (Google-native), Fivetran, or Debezium gives real-time freshness and exact delete capture. Use it only if you truly need sub-second latency. See our CDC tools guide.

BigQuery-specific gotchas

• Use load jobs, not streaming inserts. Batch load jobs are free and efficient; the streaming API costs per row and is meant for real-time event ingestion, not table replication. ingestr uses load jobs.
• Partition and cluster raw tables. BigQuery bills by bytes scanned. Partition by an ingestion date or an event date and cluster on common filter keys so downstream queries stay cheap.
• Get location right the first time. A dataset's region (US, EU, etc.) is fixed at creation. Match it to your data-residency needs; you cannot move it later without recreating and reloading.
• Land in a raw dataset. Keep replicated tables separate from modeled ones so a reload never overwrites your models.
• Mind on-demand vs slots. If replication feeds heavy transformations, on-demand pricing can spike. Partitioning and incremental models keep scanned bytes down.

After replication: model and monitor

Raw tables in BigQuery are step one; you still need to transform, quality-check, and schedule. ingestr is the ingestion layer of Bruin, an open-source platform that runs SQL/Python transformations, data quality checks, and scheduling against BigQuery in the same project as ingestion, so you are not stitching a replication tool to a separate transformation tool and scheduler.

FAQ

How do I move data from Postgres to BigQuery?

Use Bruin's open-source ingestr CLI: give it the Postgres connection as the source and your BigQuery project and dataset as the destination, name the table, and set an incremental key so subsequent runs append only new or changed rows. To replicate many tables on a schedule, define each copy as a Bruin pipeline asset so the loads, the downstream SQL models, and the quality checks run in one pipeline. Managed options are Fivetran, Airbyte Cloud, and Google's Datastream; Datastream or Debezium fit when you need true CDC.

How do I load data into BigQuery with SQL and Python?

In Bruin a pipeline mixes both: ingestr or a Python asset lands the raw data, SQL assets model it inside BigQuery, and Python assets handle anything SQL cannot, with dependencies and quality checks declared next to each asset. That is the pattern in the BigQuery pipeline use cases. dbt plus a separate loader is the common alternative, at the cost of two tools and an orchestrator to connect them.

What is the best tool to replicate a database into BigQuery?

Code-first and open source: Bruin (ingestr for the copy, the Bruin CLI for scheduling, modeling, and checks). Fully managed: Fivetran, Hevo, or Google Datastream. Low-latency CDC: Datastream, Estuary Flow, or Debezium. The sections above cover the BigQuery-specific gotchas around partitioning, streaming inserts, and cost.

Related: replicate into Snowflake or Databricks, and if your source is Firebase, our guide to exporting Firebase data to BigQuery. See also the best data ingestion tools in 2026.

I work at Bruin, which makes ingestr and Bruin. Corrections welcome at [email protected].