How do I keep full history with ingestr?
Pass the scd2 strategy to ingestr to track history with slowly changing dimensions. One flag changes how the data lands, with no custom merge code to write.
Command
ingestr ingestDefined in
CLI
Works with
Any ingestr source and destination
What you get
Incremental strategiesone flagno merge code
How it works in code
$ ingestr ingest \
--source-uri $SRC --dest-uri $DST \
--incremental-strategy scd2Run ingestr ingest and ingestr applies the scd2 strategy on every run.
Related use cases
Incremental loading strategies
Load data with the replace strategy in ingestr
How do I fully reload a table on each run with ingestr?
Incremental loading strategiesLoad data with the append strategy in ingestr
How do I append only new rows with ingestr?
Incremental loading strategiesLoad data with the merge strategy in ingestr
How do I upsert changed rows with ingestr?
Move data with one command
Open source. Source URI in, destination URI out, incremental by default.