Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions benchmarks/pm-api-latency.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,8 +41,8 @@ abstract: |
this API up" is answered by direct measurement from three regions rather
than by user reports. Samples that fail because our pinned market
expired are classified probe_invalid and never count against the venue.
Third party data aggregators that resell venue prices (Mobula, Codex,
Predexon) are tracked separately in the pm-data-freshness bench and on
Third party data aggregators that resell venue prices (Codex, Predexon)
are tracked separately in the pm-data-freshness bench and on
the data feeds tab of the prediction markets hub. This page ranks the
five venues' own APIs only, so the leaderboard answers one clean
question: which prediction market venue API is fastest right now.
Expand All @@ -56,7 +56,7 @@ methodology:
- "Division of labor with the sibling benches: how each venue behaves as request rates climb, including throttle onset and 429 handling, is measured in pm-rate-limits (bench 037). How fresh third party data providers relay Polymarket data is measured in pm-data-freshness (bench 032). This bench ranks the venues' own APIs on latency and uptime at a polite request rate."
- "Myriad's origin is a single region US East deployment behind Heroku, so its latency from eu-west and sgp is dominated by geography. Reported as measured, called out per region."
- "All five venues are probed by the same open source harness as bench 037, from the same processes, so the two benches share one probe budget and one identifying User-Agent: `OpenChainBench/1.0 (+https://openchainbench.com/methodology; contact@mobula.io)`. We publish latency and uptime measurements only, never market data."
- "Aggregators that resell venue data (Mobula, Codex, Predexon) are deliberately excluded from this leaderboard. They are not venue APIs, they are relays on top of venue APIs, and ranking them next to the venues they relay would compare two different products on one axis. Their freshness lag against the Polymarket CLOB T0 stream is the relevant metric, measured in the pm-data-freshness bench. The prediction markets hub data feeds tab lists each aggregator with the venues it covers."
- "Aggregators that resell venue data (Codex, Predexon) are deliberately excluded from this leaderboard. They are not venue APIs, they are relays on top of venue APIs, and ranking them next to the venues they relay would compare two different products on one axis. Their freshness lag against the Polymarket CLOB T0 stream is the relevant metric, measured in the pm-data-freshness bench. The prediction markets hub data feeds tab lists each aggregator with the venues it covers."
- "Regions: us-east, eu-west, sgp (Railway). Histogram buckets 25ms to 10s."

findings:
Expand All @@ -65,7 +65,7 @@ findings:
- "{{name:manifold}} would look faster than it is if cache hits counted: its whole API sits behind a 5 second CDN cache. With cache hits excluded its origin answers at {{p50:manifold}} p50, an honest number a trading bot polling fresh quotes will actually see."
- "{{name:myriad}} serves every region from a single US East origin, so its cohort worst p50 of {{p50:myriad}} is mostly geography. Switch the region dimension to us-east to see the API itself rather than the speed of light."
- "{{name:polymarket}} answers its midpoint endpoint at {{p50:polymarket}} p50. Because we probe it every 5 seconds from three regions, the uptime panel doubles as a Polymarket API status check that updates continuously instead of waiting for user reports."
- "For builders who consume venue data through a managed relay, the relevant question is freshness lag, not request latency. Mobula, Codex and Predexon are measured in the pm-data-freshness bench (T0 against the Polymarket CLOB stream). The prediction markets hub data feeds tab lists each aggregator with the venues it covers."
- "For builders who consume venue data through a managed relay, the relevant question is freshness lag, not request latency. Codex and Predexon are measured in the pm-data-freshness bench (T0 against the Polymarket CLOB stream). The prediction markets hub data feeds tab lists each aggregator with the venues it covers."

disclaimer: "Uptime here means our probes succeeded from three specific regions at a polite request rate. A venue can be up for us and degraded for you, especially during regional network incidents, and a brief blip between probe cycles can go unrecorded. Treat the panel as a measured signal, not a guarantee."

Expand All @@ -83,7 +83,7 @@ faq:
- q: "Why do latency numbers exclude CDN cache hits?"
a: "Because a cache hit measures the distance to the nearest edge node, not the venue's API. Manifold serves its entire API behind a 5 second cache and Kalshi's market list sits on CloudFront, so a naive average would mostly rank CDN proximity. Every sample carries a cache flag read from the response headers, and the headline aggregates only non cached, successful requests. The number you see is what a client polling for fresh quotes actually experiences at the origin."
- q: "Does this benchmark cover throttling or data providers?"
a: "No, deliberately. What happens to each venue's API as request rates climb, including 429 onset and queueing behaviour, is its own benchmark, pm-rate-limits, which runs a careful daily ramp. And if you consume venue data through a provider such as Mobula or Codex rather than calling the venue directly, the relevant number is relay freshness, measured in pm-data-freshness. This page answers two narrower questions: which venue API is fastest at a polite request rate, and is it up right now."
a: "No, deliberately. What happens to each venue's API as request rates climb, including 429 onset and queueing behaviour, is its own benchmark, pm-rate-limits, which runs a careful daily ramp. And if you consume venue data through a provider such as Codex or Predexon rather than calling the venue directly, the relevant number is relay freshness, measured in pm-data-freshness. This page answers two narrower questions: which venue API is fastest at a polite request rate, and is it up right now."

per_chain_explainer:
- slug: polymarket
Expand Down
30 changes: 5 additions & 25 deletions benchmarks/pm-data-freshness.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ slug: pm-data-freshness
number: "032"
title: Fastest prediction market data API, live freshness across venues
seo_title: "Fastest prediction market data API 2026"
seo_description: "Fastest prediction market data API ranked live across Polymarket and Kalshi. Milliseconds Mobula and Codex lag the venue gateway publish on top markets."
seo_description: "Fastest prediction market data API ranked live across Polymarket and Kalshi. Milliseconds Codex and Predexon lag the venue gateway publish on top markets."
subtitle: Per event delay between provider arrival and the venue gateway publish, measured every minute on the top markets by 24 hour volume across Polymarket and Kalshi.

category: Aggregators
Expand All @@ -22,8 +22,8 @@ seo_intro: |
a trade and when the provider relays the same trade to its WebSocket
subscribers. This benchmark measures exactly that. The harness
subscribes to the same basket of top volume markets on each venue's own
canonical source (T0), on Mobula's PM WebSocket, and on Codex GraphQL
subscriptions. Each trade is cross correlated across the streams, and
canonical source (T0) and on Codex GraphQL subscriptions. Each trade
is cross correlated across the streams, and
the per provider lag versus the venue gateway publish time is recorded
as a Prometheus histogram. The leaderboard sorts by p50 freshness delta
in milliseconds, lower is better. Use the venue tab at the top of the
Expand All @@ -41,14 +41,12 @@ abstract: |
seconds are not counted toward their p50, only toward their receive
total, so a provider can look fresh on the leaderboard while silently
dropping events. The success rate column flags that. Coverage today:
Mobula and Codex on Polymarket, Codex only on Kalshi (Mobula does not
yet cover Kalshi venue data).
Codex on Polymarket and Kalshi.

methodology:
- "Anchor rows: on each venue tab the venue itself (Polymarket, Kalshi) is the canonical T0, so its row reads as the harness network round trip, near zero by construction. It is kept on the board to document the reference point, not as a competing feed; the comparison rows are the data providers measured against it."
- "Polymarket T0. `wss://ws-subscriptions-clob.polymarket.com/ws/market` is public, no auth, sub 50ms gateway publish latency from EU West. Cross correlation uses (conditionId, priceUSD rounded to 3 decimals, sizeUSD micros, 5s time bucket)."
- "Kalshi T0. `https://api.elections.kalshi.com/v1/social/trades` (REST), polled every 5s with cursor pagination. Every trade carries `create_date` at microsecond precision; that timestamp is the canonical T0. Poll cadence affects only correlation timing, not the freshness number. The official Kalshi WebSocket needs RSA PSS signed headers from a US KYC account and 403s from non US IPs, not viable for a public benchmark."
- "Mobula PM WebSocket. `wss://pm-api-prod-eu.mobula.io`. Auth via API key in the subscribe payload. Cloudflare on the gateway requires a browser User Agent on the upgrade request, default Go HTTP UA is silently filtered. Covers Polymarket today, Kalshi coverage is not yet shipped."
- "Codex GraphQL subscriptions. `wss://graph.codex.io/graphql` with the `graphql-transport-ws` subprotocol. Firehose `onPredictionTradesCreated` subscription filtered client side to the venue's marketIds. Covers both Polymarket and Kalshi."
- "Basket. Top 20 active markets by 24h volume per venue, refreshed every 5 minutes. Polymarket pulls from `gamma-api.polymarket.com`, Kalshi pulls from `api.elections.kalshi.com/trade-api/v2/markets`."
- "Histogram buckets: 10, 25, 50, 100, 250, 500, 1000, 2500, 5000, 10000, 30000, 60000 ms. The venue's own arrivals always land in the smallest bucket because by construction we measure other providers against it."
Expand All @@ -57,21 +55,16 @@ methodology:

findings:
- "Each venue's own gateway is by definition the freshest source on the leaderboard for that venue. The lag versus its own publish time is on the order of the network round trip from the harness to the gateway, typically below 100 ms p50 from EU West."
- "{{name:mobula}} relays Polymarket trades with a p50 delta of {{p50:mobula}} versus the venue native gateway on the Polymarket tab. Mobula's PM WebSocket runs on a dedicated edge service in EU West and forwards events without orderbook reconstruction, which is why the gap to native is small. Mobula does not yet cover Kalshi, so the Kalshi tab shows Codex alongside the Kalshi T0 row only."
- "{{name:codex}} sits at p50 {{p50:codex}} on the Polymarket tab because Codex indexes the chain event (Polygon block confirmation) rather than the off chain orderbook publish. The lag includes Polygon block time (~2s) plus ingestion. On Kalshi Codex ingests Kalshi's own WebSocket feed directly, so the delta is purely pipeline latency."
- "The spread between providers reflects integration depth: native gateway vs edge cached relay vs chain indexed pipeline. None of these is wrong, they answer different questions. For live trading UIs the gateway path is the only viable one."
- "The spread between providers reflects integration depth: native gateway vs chain indexed pipeline. None of these is wrong, they answer different questions. For live trading UIs the gateway path is the only viable one."

faq:
- q: "Which prediction market data API has the lowest latency right now?"
a: "{{best_name}} currently leads at {{best_p50}} (p50 over the last 24h) measured as time from the venue's own canonical WebSocket publish to provider relay. The leaderboard re sorts every minute on fresh Prometheus samples, so the answer reflects the actual measured lag on the active market basket, not a marketing claim. Switch the venue tab at the top to see Polymarket vs Kalshi independently."
- q: "What does 'freshness delta' mean for a prediction market API?"
a: "We connect to the venue's own canonical WebSocket and to the provider's WebSocket simultaneously, subscribe to the same markets, and for every trade event we record how many milliseconds the provider takes to relay the event after the venue itself publishes it. Lower is better. The venue's own gateway publish time is the canonical T0 because by construction nothing downstream can be faster than the source."
- q: "Is Mobula's PM WebSocket faster than Codex?"
a: "When the Polymarket tab is selected, it depends on what each provider does under the hood. {{name:mobula}} is an edge cached relay of Polymarket's own gateway, so the p50 delta is roughly the network round trip between the two gateways plus a few ms of bookkeeping. {{name:codex}} ingests the on chain confirmation on Polygon, which adds the block time (~2 seconds) before any trade can be relayed. For a live UI building on Polymarket, the relay path wins on freshness. For on chain reconciliation or settlement workflows, the chain indexed path is what you actually want. On the Kalshi tab the comparison does not apply yet because Mobula does not cover Kalshi."
- q: "Why don't you include Polymarket REST polling on this benchmark?"
a: "When the Polymarket tab is selected, freshness is a WebSocket question. REST polling at 1s would have a floor freshness around 500ms (poll interval / 2) plus RTT, dominated by how often you poll. The Polymarket gateway WebSocket exists for exactly this reason, to avoid that floor. Adding REST as a row would make the leaderboard noisy without changing the conclusion: WebSocket beats polling by definition for real time data."
- q: "Why is only Codex shown on the Kalshi tab?"
a: "Because Mobula's PM API does not cover Kalshi venue data today. The Kalshi tab compares Codex against the Kalshi T0 source only. Mobula coverage of Kalshi is on the roadmap; when it ships the same provider will appear on both tabs and the comparison becomes apples to apples again."
- q: "What does the Kalshi T0 source measure?"
a: "Kalshi's own public WebSocket at `wss://external-api-ws.kalshi.com/trade-api/ws/v2`, channel `trade`. The harness reads the `msg.ts_ms` field of each fill as the canonical publish timestamp and uses it as T0 for every Kalshi side comparison. Just like Polymarket's CLOB WebSocket on the Polymarket tab, it is the source against which provider latency is measured."
- q: "Are these numbers comparable to Kalshi or Limitless?"
Expand Down Expand Up @@ -131,19 +124,6 @@ providers:
sample_size: sum(increase(pm_events_total{provider="kalshi"}[24h]))
series: pm_health{provider="kalshi"} * 0.5

- slug: mobula
name: Mobula
tag: Edge cached venue relay, browser UA required (Polymarket only today)
formula: "Median ms lag versus the venue gateway, measured by cross correlating trades over the rotating basket of top 20 markets. Polymarket coverage today, Kalshi pending."
queries:
p50: histogram_quantile(0.50, sum(rate(pm_freshness_delta_ms_bucket{provider="mobula",kind="trade"}[24h])) by (le))
p90: histogram_quantile(0.90, sum(rate(pm_freshness_delta_ms_bucket{provider="mobula",kind="trade"}[24h])) by (le))
p99: histogram_quantile(0.99, sum(rate(pm_freshness_delta_ms_bucket{provider="mobula",kind="trade"}[24h])) by (le))
mean: sum(rate(pm_freshness_delta_ms_sum{provider="mobula",kind="trade"}[24h])) / sum(rate(pm_freshness_delta_ms_count{provider="mobula",kind="trade"}[24h]))
success: clamp_max(sum(rate(pm_matched_total{provider="mobula",kind="trade"}[24h])) / sum(rate(pm_events_total{provider="polymarket",kind="trade"}[24h])), 1)
sample_size: sum(increase(pm_matched_total{provider="mobula",kind="trade"}[24h]))
series: histogram_quantile(0.50, sum(rate(pm_freshness_delta_ms_bucket{provider="mobula",kind="trade"}[1h])) by (le))

- slug: codex
name: Codex
tag: Chain indexer on Polymarket, native ingestion on Kalshi
Expand Down
2 changes: 1 addition & 1 deletion benchmarks/pm-rate-limits.yml
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ faq:
- q: "How do you test rate limits without abusing the APIs?"
a: "Four guardrails. Tiers stay well inside documented budgets where they exist (at most 7 percent of Polymarket's book allowance). Every run aborts as soon as throttled plus server errors exceed 1 percent of a 10 second window, and Kalshi stops at the first 429. Only one region ramps a venue at a time, on disjoint UTC hours. And every request carries an identifying User-Agent with a contact address, so a venue can reach us or filter us selectively. The measurement is throttle onset, not stress to failure."
- q: "Why measure the venue APIs directly instead of a data provider?"
a: "They answer different questions. If you build directly on a venue, its native API latency and throttle behaviour set your floor, and that is what this benchmark measures. If you consume the venue through a data provider such as Mobula or Codex, what matters is how fresh the provider's relay is, which is measured separately in the pm-data-freshness benchmark. Both pages link each other so you can compare the paths."
a: "They answer different questions. If you build directly on a venue, its native API latency and throttle behaviour set your floor, and that is what this benchmark measures. If you consume the venue through a data provider such as Codex or Predexon, what matters is how fresh the provider's relay is, which is measured separately in the pm-data-freshness benchmark. Both pages link each other so you can compare the paths."

per_chain_explainer:
- slug: polymarket
Expand Down
4 changes: 3 additions & 1 deletion src/lib/pm-stats.ts
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,9 @@ const PM_VENUES: VenueSeed[] = [
];

const PM_DATA_FEEDS: DataFeedSeed[] = [
{ slug: "mobula", name: "Mobula", coverage: ["polymarket"], isReference: false },
// Mobula dropped 2026-07-19: they stopped serving the PM WebSocket
// relay, so the hub cohort and pm-data-freshness bench no longer
// rank a dead endpoint.
{ slug: "codex", name: "Codex", coverage: ["polymarket", "kalshi"], isReference: false },
{ slug: "predexon", name: "Predexon", coverage: ["polymarket", "kalshi", "limitless"], isReference: false },
];
Expand Down
1 change: 0 additions & 1 deletion src/lib/pm-venue-context.ts
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,6 @@ export const PM_VENUE_META: Record<string, { url: string; chainLabel: string }>
// mobula, and predexon.
export const PM_FEED_META: Record<string, { url?: string }> = {
"polymarket-clob": { url: "https://polymarket.com" },
mobula: { url: "https://mobula.io" },
codex: { url: "https://codex.io" },
predexon: { url: "https://predexon.com" },
};
Expand Down
4 changes: 2 additions & 2 deletions src/lib/spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -297,7 +297,7 @@ const loadBenchmarkUnfilteredCached = unstable_cache(
// v31: 084 indexer-latency dropped entirely (spec + harness deleted;
// HyperSync + The Graph required paid credentials for sustained
// cadence). Bench SET shrank.
["bench-unfiltered-v35", process.env.VERCEL_ENV === "production" ? "prod" : "all"],
["bench-unfiltered-v36", process.env.VERCEL_ENV === "production" ? "prod" : "all"],
{ revalidate: 300, tags: ["benchmarks"] },
);

Expand Down Expand Up @@ -456,7 +456,7 @@ const loadAllBenchmarksCached = unstable_cache(
// v32: bumped with bench-unfiltered-v30 (081 slug rename ws-head-latency
// -> ws-head-latency-ethereum + ungate on prod).
// v34: bumped with bench-unfiltered-v31 (084 indexer-latency dropped).
["all-benchmarks-v38", process.env.VERCEL_ENV === "production" ? "prod" : "all"],
["all-benchmarks-v39", process.env.VERCEL_ENV === "production" ? "prod" : "all"],
{ revalidate: 300, tags: ["benchmarks"] },
);
export const loadAllBenchmarks = cache(loadAllBenchmarksCached);
Expand Down
Loading