B3 Options

B3 Options Intelligence is an end-to-end data pipeline for Brazilian listed options monitoring, built with Bruin, Python, and DuckDB. B3 distributes daily and historical market data as compressed fixed-width files — one per session, with exchange-specific record layouts for equities, options, futures, and forwards mixed in the same file. There is no API. The starting point is a ZIP, the format is a 245-character-wide flat file, and the option classification requires cross-referencing market type codes with ticker suffix patterns to isolate standard equity options. This pipeline handles all of that and builds a monitoring layer on top. Pipeline: Python assets download, extract, and parse COTAHIST files using pandas.read_fwf with exchange-defined column specs Idempotent ingestion with deterministic row hashing across both daily and annual historical files Staging layer merges and deduplicates yearly and daily sources, classifies option type and underlying ticker, and serves as the single clean foundation for all marts 8 mart tables power the full monitoring workflow: unusual activity (volume and trade count Z-scores), liquidity surface by expiry and moneyness bucket, put/call flow regime, expiry concentration, dominant call/put walls per maturity, strike ladder, contract-level daily, and underlying-level daily with composite volume spike Streamlit dashboard for visual exploration with OHLC overlays from the underlying equity data GitHub Actions CI running bruin validate . on every push Numbers: backfilled from 2015, ~15K rows per daily session, 8 mart assets + 2 staging views, all checks passing. Why Bruin: ingestion is procedural Python, modeling is declarative SQL. Forcing either into the wrong layer would make the pipeline worse. Bruin handles both in a single DAG with dependency resolution, data quality checks, and column-level metadata — without a separate orchestrator.