Back to module
Step 3-12 min

Ingest FMP, Yahoo Finance, and FRED

Have the AI agent write ingestr assets for three data sources - company fundamentals, stock prices, and US macro - and pull them into DuckDB.

What you'll do

Get free API keys for FMP and FRED (Yahoo Finance needs none), add them to your project config, and have the AI agent scaffold three ingestr assets. Then run the pipeline once to populate DuckDB.

Why this step matters

This is the entire data-import layer of a stock analyst's stack, in one step:

  • FMP gives you quarterly financials and ratios - the numbers every analyst looks at first
  • Yahoo Finance gives you daily price bars (open / high / low / close / volume) - how the market has priced the company
  • FRED gives you the macro backdrop - rates, inflation, unemployment, the yield curve - the regime your picks live inside of

Most beginners stop at one source and build shallow analysis. With all three in the same database you can ask questions that combine them, which is where the real signal lives.

1. Get your free API keys

Two signups, 1 minute each. Yahoo Finance needs no key.

Keep both keys in a note somewhere. You'll paste them into .bruin.yml in the next step.

2. Add the source connections

Prompt the agent:

AI Prompt

Add three new connections to .bruin.yml under environments.default.connections:

  1. An fmp connection named fmp-default with API key <YOUR_FMP_KEY>
  2. A yahoo_finance connection named yahoo-default (no credentials needed)
  3. A fred connection named fred-default with API key <YOUR_FRED_KEY>

Use the ingestr docs at https://bruin-data.github.io/ingestr/supported-sources.html for the exact field names. Then run bruin connections test for each and show me the output.

Don't want to hand the agent your keys? Ask it instead to scaffold the connections in .bruin.yml with placeholder values (e.g. <YOUR_FMP_KEY>, <YOUR_FRED_KEY>). Then open .bruin.yml yourself and paste the real keys in. The file is git-ignored by default, so they stay local.

When the agent finishes, .bruin.yml will have four connections total: your DuckDB destination from Step 2 plus the three new sources.

Security note. .bruin.yml is added to .gitignore by default - your keys won't end up in git. If you're sharing the project, use environment variables instead; the docs show how.

3. Have the agent write three ingestr assets

Now the good part. Prompt:

AI Prompt

In stock-analyst-101/assets/, create three ingestr assets using the ingestr documentation as reference:

fmp_fundamentals.asset.yml - pull quarterly income statement, balance sheet, and cash flow for tickers AAPL, MSFT, NVDA, GOOGL, AMZN. Destination: duckdb-default, schema raw, table fmp_fundamentals. Use merge strategy on symbol + date.

yahoo_prices.asset.yml - pull daily OHLCV prices for the same 5 tickers from 2020-01-01 to today. Destination: duckdb-default, schema raw, table yahoo_prices. Use merge strategy on symbol + date.

fred_macro.asset.yml - pull these FRED series: FEDFUNDS (Fed funds rate), CPIAUCSL (CPI), UNRATE (unemployment), T10Y2Y (10Y-2Y yield spread), GDPC1 (real GDP). Destination: duckdb-default, schema raw, table fred_macro. Use merge strategy on series_id + date.

Before writing the files, show me each YAML and explain what it does. Don't run anything yet.

The agent will produce three YAML files. Read them - they should look roughly like:

name: raw.fmp_fundamentals
type: ingestr
parameters:
  source_connection: fmp-default
  source_table: "income-statement,balance-sheet-statement,cash-flow-statement"
  destination: duckdb
  destination_connection: duckdb-default
  incremental_strategy: merge
  incremental_key: "symbol,date"
  interval_start: "2020-01-01"
  tickers: "AAPL,MSFT,NVDA,GOOGL,AMZN"

Field names vary by source - the ingestr docs are the source of truth. Trust what the agent writes but glance at the YAML to make sure the destination, schema, and table name match what you asked for.

4. Run the pipeline

One more prompt:

AI Prompt

Run bruin run stock-analyst-101/ and stream the output. Tell me which assets succeeded and any that failed.

This kicks off all three ingestions in parallel. First run can take 2–5 minutes depending on how much history Yahoo returns. When it finishes, you'll have three tables in raw.* inside stock.duckdb.

5. Peek at what landed

Prompt:

AI Prompt

Using bruin query, show me SELECT COUNT(*), MIN(date), MAX(date) FROM raw.yahoo_prices GROUP BY symbol. Do the same shape of count for raw.fmp_fundamentals and raw.fred_macro.

You should see a few thousand price rows per ticker, ~20 fundamental rows per ticker (quarterly history), and one row per (series × date) for the macro table.

Troubleshooting

  • FMP "API limit reached" - the free plan is generous but rate-limited. Wait a minute and re-run.
  • Yahoo Finance "No data for ticker" - check the ticker symbol. BRK.B needs to be BRK-B for Yahoo.
  • FRED "Invalid API key" - keys are case-sensitive and have no spaces. Re-copy from fredapi settings.
  • DuckDB file locked - you have DuckDB open in another process (maybe a desktop app). Close it, re-run.

What just happened

You just built a three-source financial data stack on a laptop, with zero SQL written by you. Every byte of it lives in stock.duckdb - one file, portable, inspectable with any SQL tool. Next step: teach the AI agent what these tables mean so it can reason about them accurately.