feat(kalshi): add market_details and market_trades gold layer

[los-xyz]

Summary

Adds two new Kalshi prediction market spells, transforming API-sourced bronze tables into a clean gold layer:

• kalshi.market_details (TABLE): Market reference table joining markets_raw with event metadata from market_details_raw. Filtered to markets with >= 100 contracts traded.
• kalshi.market_trades (VIEW): Trade-level table enriched with market metadata via inner join to market_details.

Design choices (bronze → gold)

• >= 100 contracts filter: drops 85% of markets (dust/empty), keeps 99.7% of volume
• 12 columns dropped: universally null (functional_strike, mve_*, is_provisional, fee_waiver_expiration_time), constant (response_price_units, notional_value_dollars, price_ranges), always zero (liquidity_dollars), borderline sparse (rules_secondary, primary_participant_key, settlement_timer_seconds), internal (created_hour)
• INNER join on trades: ensures only trades for meaningful markets flow to gold layer
• Pricing snapshot kept: despite being latest-state-only, the orderbook/OI columns are well-populated on active markets and useful for current-state analysis

Test plan

• dbt compile passes for both models
• CI builds and tests pass
• Verify market_details unique on ticker
• Verify market_trades unique on trade_id
• Spot-check join completeness (NULL rates on enriched columns)

🤖 Generated with Claude Code

[jeff-dude]

pushed changes to make the models incremental, since larger tables.
would be good if you could test the output build in CI here. note that CI tables get dropped over the weekend, so you may need to rerun the actions to rebuild if looking later.
plz ensure these outputs still match what you expect

[cursor]

PR Summary

Medium Risk
Adds new incremental/merge dbt models and sources for Kalshi, including join logic and rolling-window incremental predicates that can impact backfill size and correctness if source timestamps or keys behave unexpectedly.

Overview
Introduces a new Kalshi gold layer with two incremental merge models: kalshi.market_details (market reference data) and kalshi.market_trades (trade facts enriched with market metadata).

market_details joins markets_raw with market_details_raw, filters to markets with volume_fp >= 100, and adds a watermark_ts to drive incremental refreshes; market_trades is partitioned by block_month, merges on (block_month, trade_id), and reprocesses trades for tickers whose market_details.watermark_ts changed to keep dimension fields current.

Also registers three new Kalshi raw sources (markets_raw, market_details_raw, market_trades_raw) and adds schema docs/tests enforcing non-null and uniqueness constraints for the new models.

^{Reviewed by Cursor Bugbot for commit 4f4ca39. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Bugbot Autofix prepared a fix for the issue found in the latest run.

• ✅ Fixed: Missing deduplication on event_details CTE causes row fan-out
  • Added QUALIFY with ROW_NUMBER() to event_details CTE to keep only the latest row per event_ticker, preventing join fan-out and duplicate ticker rows.

Or push these changes by commenting:

@cursor push 491ff46c63

Preview (491ff46c63)

diff --git a/dbt_subprojects/daily_spellbook/models/_projects/kalshi/kalshi_market_details.sql b/dbt_subprojects/daily_spellbook/models/_projects/kalshi/kalshi_market_details.sql
--- a/dbt_subprojects/daily_spellbook/models/_projects/kalshi/kalshi_market_details.sql
+++ b/dbt_subprojects/daily_spellbook/models/_projects/kalshi/kalshi_market_details.sql
@@ -54,6 +54,7 @@
 		strike_period,
 		last_updated_ts
 	from {{ source('kalshi', 'market_details_raw') }}
+	qualify row_number() over (partition by event_ticker order by last_updated_ts desc) = 1
 )
 
 select

_{You can send follow-ups to the cloud agent here.}

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 4f4ca39. Configure here.}