How do I build a context layer so LLMs query my DuckDB data correctly?
Bruin parses the SQL in each DuckDB asset and builds column-level lineage with no manual mapping, then exposes lineage, column descriptions, and checks to agents through its MCP server. Because it comes from parsing your project rather than from warehouse query logs, the lineage exists before the code runs, which is what lets bruin validate fail a pull request that would break something downstream. A catalog like OpenMetadata or Atlan covers more ground across tools Bruin does not run, but reconstructs lineage after the query has already executed.
Command
bruin lineageDefined in
SQL + YAML
Works with
DuckDB + Bruin CLI
What you get
How to do it
- 1
Point Bruin at your DuckDB project so it can parse every asset.
- 2
Run bruin lineage ./pipeline --asset <name> to see the graph for one asset.
- 3
Add descriptions to the columns people actually ask about, in the asset definition.
- 4
Run bruin validate to confirm every reference resolves.
- 5
Add bruin validate to CI so a breaking change fails in the pull request.
- 6
Expose the graph to agents with bruin mcp if you want AI answers grounded in it.
How it works in code
$ bruin lineage ./pipeline --asset mart.ordersRun bruin lineage and Bruin prints the column graph across your DuckDB assets.
Worth knowing
Bruin only sees assets inside a Bruin pipeline. A Looker explore, a notebook, or a legacy job touching DuckDB is invisible to it, so if coverage across tools you do not control is the requirement, use a catalog instead.
Other ways to do this
Bruin is not always the right answer. Here is where the alternatives are stronger.
| Option | When it is the better choice |
|---|---|
| Bruin | Use column-level lineage parsed from your DuckDB pipeline SQL, available before a change merges rather than after it runs. |
| OpenMetadata | Open-source catalog with column-level lineage across many tools, not just DuckDB assets you define in one framework. Heavier to operate. |
| Atlan | The strongest commercial catalog for organisation-wide governance with business stakeholders, glossary, and stewardship workflow. A different product category to a CI gate. |
| dbt | Free model-level lineage if your transformations are already in dbt. Column-level resolution needs a catalog or a paid tier. |
Common questions
How do I build a context layer so LLMs query my DuckDB data correctly?
Bruin parses the SQL in your DuckDB assets to build column-level lineage, then exposes lineage, column descriptions, and checks to agents through its MCP server. Run bruin lineage against an asset to see it.
Does DuckDB lineage require manual mapping in Bruin?
No. The lineage is derived from parsing the SQL in each asset, so it updates when the code does and cannot drift the way a hand-maintained catalog entry does.
How is this different from a data catalog on DuckDB?
A catalog reconstructs lineage from warehouse query logs, which covers every tool that touched the warehouse but only after queries have run. Parsing the project gives lineage before the change ships, which is what makes a pre-merge CI gate possible. Large organisations usually want both.
Related use cases
Build a context layer on Snowflake for LLM agents
How do I build a context layer so LLMs query my Snowflake data correctly?
AI context by warehouseBuild a context layer on BigQuery for LLM agents
How do I build a context layer so LLMs query my BigQuery data correctly?
AI context by warehouseBuild a context layer on Databricks for LLM agents
How do I build a context layer so LLMs query my Databricks data correctly?
Know what breaks before you ship it
Open source. Column-level lineage parsed from your SQL, with no manual mapping.