Back to Showcase

NZ Electricity Generation Pipeline

This project tracks New Zealand's electricity generation mix across 8 years (2018–2026), pulling monthly CSVs from the Electricity Authority's public API through a three-layer transformation pipeline (staging, core, mart) that feeds a Looker Studio dashboard, revealing an ~85% renewable grid driven mostly by hydro. The pipeline is built entirely with Bruin, an open-source CLI tool that replaces the usual Airflow + dbt + Great Expectations stack with a single binary: SQL and Python assets coexist in the same pipeline with automatic dependency resolution, incremental materialisation, and quality checks embedded directly in asset definitions rather than maintained as a separate test suite. That "one tool, one config format" design meant I could focus on the data logic, unpivoting 50 trading-period columns, deduplicating records, and building partitioned and clustered fact tables, rather than writing glue code between tools.

Certificate of Completion

BRUIN ACADEMYVerified Certificate of CompletionTHIS IS TO CERTIFY THATXinglin GaoHAS SUCCESSFULLY COMPLETED THE PROJECTEnd-to-End Data Pipeline with BruinProject: NZ Electricity Generation Pipeline · Issued May 2026VERIFIEDBRUIN ACADEMYSabri KaragonenSabri KaragonenCO-FOUNDER, BRUINArsalan NoorafkanArsalan NoorafkanDEVELOPER ADVOCATE, BRUINThis certificate has been verified. Confirm authenticity athttps://getbruin.com/project-showcase/nz-electricity-generation-pipeline/

Saves as a high-resolution PNG image.

Share: