feat: prediction-market resolution outcomes, price and trade history (Kalshi, ForecastEx, Polymarket, Polymarket US)

[stefan-jansen]

Summary

Adds what research on resolved prediction markets needs: every resolved market with a normalized outcome, price history that reaches resolved markets, and trade history where the venue publishes it for free.

Kalshi (KalshiProvider, existing methods unchanged)

• iter_markets / fetch_markets: cursor-paginated listing over the live /markets endpoint and the /historical/markets archive. /historical/cutoff decides when the archive can be skipped. Filters: status (incl. settled), series, event, min/max close time, and Kalshi's mve_filter (None includes combo/parlay KXMVE* markets, "exclude" drops them, "only" selects them; the archive rejects only, so it is applied on the client there). The archive accepts only one of event/series/mve filter and no status or time filter, so the remaining filters run on the client. Live/archive duplicates are dropped.

• Market frame: ticker, event_ticker, series, title, status, market_type, open/close time, result, settlement_value, settlement_ts, result_raw, expiration_value, last_price, previous_price, volume, open_interest, source.

• iter_trades / fetch_trades: trades for a ticker over /markets/trades and /historical/trades, with min/max time; columns include taker_side, taker_outcome_side and taker_book_side.

• Reads the *_dollars/*_fp fields and the legacy unsuffixed fields (integers as cents, strings as dollars).

• fetch_candles(ticker, start_ts, end_ts, period, series_ticker=None, tier=None): price history from whichever tier holds the market (/historical/markets/{ticker}/candlesticks for archived markets, /series/{series}/markets/{ticker}/candlesticks for live ones). Given tier or series_ticker, it goes straight to the right endpoint. Otherwise a live 404 falls back to the archive, and a second 404 looks the series up from the market's event (FED-23DEC is in series KXFED). Ranges over Kalshi's 5000-candle limit are split. Columns: trade OHLC, mean, previous, YES bid/ask close, volume, open interest, source. fetch_ohlcv keeps its signature and now uses the same routing.

• Units fix: unsuffixed integers are legacy cents; *_dollars values and unsuffixed strings (archive candles) are dollars. The old blanket /100 would have scaled archive candles down 100x.

• iter_market_frames(chunk_size=100_000, ...) streams the normalized market rows for listings too large to hold in memory; fetch_markets delegates to it.

ForecastEx (new ForecastExProvider)

• Reads the free daily CSVs in the public forecastex-public-data S3 bucket: fetch_prices, fetch_trades (every matched pair), fetch_products.
• fetch_markets derives the outcome. A contract that is missing from the next day's prices file has expired, and a final YES settlement price of 1.00 or 0.00 is the payout. Intraday contracts (hourly temperatures) drop out of the files with a daily mark, not their payout. They are reported as expired with a null result instead of a guessed outcome.

Polymarket US (new PolymarketUSProvider, separate from the global venue below)

• Offset-paginated /v1/markets (the server caps pages at 500), /v1/markets/{slug}/settlement, and /v1/price-history (full history, or 24-hour one-minute windows).
• The outcome comes from the long market side. On this gateway the outcomes labels are sometimes listed in a different order from outcomePrices: ["No","Yes"] comes with long-first ["1","0"] for the FOMC 2026-04-29 "maintains" market, whose YES side won. Zipping the two lists gives the wrong winner.

Polymarket, global venue (PolymarketProvider, existing methods kept)

• iter_markets / iter_market_frames(chunk_size, closed=True, min_volume=None, end_date_min=None, end_date_max=None) / fetch_markets: every market through Gamma's GET /markets/keyset (GET /markets refuses offsets above 2000). The cursor goes back as after_cursor; any other parameter name silently returns page 1, so a repeated cursor raises. Pages hold at most 100 markets. The server honours closed, volume_num_min, end_date_min and end_date_max (verified 2026-10-08), and the client applies them again. include_tag=true brings the market tags; category is mostly null on recent markets.
• Market frame: ticker (condition id), market_id, slug, question, event slug/title, category, tags, yes/no outcome labels and token ids, closed, open/close time, result, settlement_value, settlement_ts, uma_resolution_status, outcome_prices, volume, neg_risk, neg_risk_market_id and the raw fee fields. The first outcome is the YES side whatever its label (["Trump","Boden"]: Trump is YES). Final prices [1,0]/[0,1] give yes/no, [0.5,0.5] gives void, a UMA-resolved market with other prices gives other, and everything else is null. settlement_ts is closedTime, else umaEndDate.
• fetch_candles(token_id, start, end, fidelity_minutes): timestamp and price. interval=max returns an empty history for resolved markets at fidelities finer than one day, and startTs/endTs spans over 15 days get HTTP 400. Requests therefore send explicit bounds in windows of at most 15 days, which serves resolved markets down to one-minute fidelity.
• iter_trades / fetch_trades(condition_id, start=None, end=None): data-API /trades, which returns trades newest first with inclusive start/end bounds, at most 10,000 per request, and rejects offsets above 10,000. Past the offset cap, end moves to the oldest second seen and trades from that second that were already yielded are dropped. A repeated page, or a single second with more trades than one offset window reaches, raises. price is the YES price (one minus the token price for the second outcome) and taker_side is yes/no, from outcome index and BUY/SELL.
• Fix: fetch_ohlcv returned no rows for past ranges, because it sent interval (a range ending now, which overrides the bounds) together with startTs/endTs. It also read YYYY-MM-DD bounds in the machine's local time zone.

Shared

• prediction_markets.py: the outcome vocabulary (result is one of yes, no, void, other, or null while unresolved) plus settlement_value and settlement_ts, and the trade schema shared by the adapters.
• BaseProvider._request / _request_json: a rate-limited GET that maps 429 / 404 / 5xx / transport failures to the shared exceptions and retries transient failures under the existing retry policy, which is now exposed as provider_retry and honors Retry-After.
• Both new providers are registered as events providers. They are not routed through DataManager.

Dependencies

• uv.lock: multidict 6.9.1, urllib3 2.8.0, virtualenv 21.14.6. The CI dependency audit started flagging the previously locked versions after main's last run, and the audit job fails until they are bumped.

Tests

• Offline: tests/providers/test_kalshi_history.py, test_forecastex_provider.py and test_polymarket_us_provider.py run against in-process httpx.MockTransport venues. They cover success, pagination within and across the live/archive boundary, empty responses, malformed JSON/CSV/records, 429 retry honoring Retry-After, persistent 429, 5xx, transport errors, and repeated cursors.
• Each venue has a resolution-outcome test that fails when the outcome is dropped. Checked by mutation: replacing the outcome with null, ignoring the next-day absence rule for ForecastEx, or zipping Polymarket US labels with prices each fails the test.
• tests/providers/test_polymarket_history.py replays Gamma, CLOB and data-API responses recorded on 2026-10-08 (tests/providers/fixtures/polymarket/) through fake servers that implement the verified paging, window and offset rules. Each test names the defect it prevents. Every test was shown to fail with its defect planted (20 mutations, including a wrong cursor parameter, removed repeated-cursor and repeated-page guards, a 30-day price window, interval=max, a raw NO-token price reported as YES, and the boundary second dropped or duplicated when paging trades). The fetch_ohlcv tests fail on the old interval request and, under a non-UTC TZ, on local-zone dates.
• Live smoke tests (integration marker): tests/integration/test_kalshi.py::TestKalshiResolvedHistory (gated by RUN_PUBLIC_KALSHI_TESTS=1), tests/integration/test_forecastex.py, tests/integration/test_polymarket_us.py and tests/integration/test_polymarket.py::TestPolymarketResolvedHistory. All passed on 2026-10-08, including combo-market selection, hourly candles for one archived and one live KXFEDDECISION market, and 20 days of hourly Polymarket prices plus trades for a market resolved on 2026-01-01. The public job in provider-contracts.yml runs these tests, and tests/test_release_workflows.py checks that it does.

Gates run locally

ruff check, ruff format --check, ty check, pytest -W error::ResourceWarning (3759 passed), mkdocs build --strict, uv build, wheel import of the new exports, actionlint, pre-commit run --all-files, pip-audit of the locked environment: all pass.

🤖 Generated with Claude Code

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.