Can Python models be first-class citizens next to SQL?
A Python asset is defined the same way as a SQL asset, with YAML metadata and dependencies. Bruin materializes its output into the warehouse.
Command
bruin runDefined in
Python + YAML
Works with
Bruin CLI + your warehouse
What you get
First-class Python assetsisolated envswarehouse materialization
How it works in code
# scores.py
""" @bruin
name: ml.scores
materialization: {type: table}
@bruin """
import pandas as pd
def materialize():
return model.predict(load_features())Run bruin run and Bruin runs the Python asset in its own environment and lands the output in the warehouse.
Related use cases
Python in pipelines
Score an ML model inside a pipeline
How do I run model scoring as part of a data pipeline?
Python in pipelinesCall an external API and land the result
Can a pipeline step call an API and store the response?
Python in pipelinesShare a Python environment across assets
How do I manage Python dependencies for pipeline steps?
SQL and Python, one pipeline
Open source. Define assets, declare dependencies, run the graph with bruin run.