Closer Every Year

A batch pipeline tracking gender gap indicators and relationship trends (marriage, divorce, age at first marriage, pay gap) across European countries from 2005 to 2024, all sourced from the Eurostat API. Features a dual-environment setup: local pipeline runs on DuckDB, GCP pipeline runs on BigQuery — same asset code, different connections. Uses Python assets for ingestion (Eurostat API → Parquet on GCS), SQL assets for staging and analytics transformations, merge strategy on (country, year) for full idempotency, and Docker-based setup with Bruin container + Terraform container side by side. CI/CD via GitHub Actions scheduled twice a year matching Eurostat update cadence.