diff --git a/benchmarks/token-trade-coverage.yml b/benchmarks/token-trade-coverage.yml new file mode 100644 index 00000000..4aabbcc2 --- /dev/null +++ b/benchmarks/token-trade-coverage.yml @@ -0,0 +1,131 @@ +# OpenChainBench. Bench № 090 + +slug: token-trade-coverage +number: "090" +title: Most complete onchain trade data API, live coverage across Solana, EVM and Stellar +seo_title: "Best token trade data API 2026 (Mobula vs Bitquery vs Codex)" +seo_description: "Which onchain data API returns the most complete trade history for a newly launched token? Live coverage % across Mobula, Bitquery and Codex on Solana, Ethereum, BSC, Base and Stellar." +subtitle: For each reference token per chain, we fetch every trade in a fixed measurement window from each provider and compare against the union baseline. Higher percent means the provider returned more of the true trade set, with fewer missed DEXs and fewer dropped fills. +category: Aggregators +status: draft +metric: Trade capture rate +unit: pct +higher_is_better: true + +seo_intro: | + This benchmark measures how much of a token's real trade history each + onchain data API actually returns. For a fixed set of reference tokens + per chain (Solana, Ethereum, BSC, Base, Stellar), the harness fetches + every trade in the same measurement window from Mobula, Bitquery, + Codex, then computes each provider's capture rate against + the union baseline (max count across the three). A rate of 100 means + the provider returned as many trades as the best-covering provider on + that token. A rate of 20 means the provider missed 80 percent of the + swaps that did happen. The comparison is chain-scoped because + coverage is not uniform: a provider strong on Solana DEX indexing can + be weak on niche BSC pools, and no provider covers every chain + equally (Stellar coverage in particular is uneven since only two of + the three measured providers index it at all today). + +abstract: | + Coverage is measured on a fixed set of reference tokens per chain, + querying each provider's public trade endpoint for the same time + window. We take the union of all trades returned across all providers + as the baseline (denominator) and each provider's returned count as + the numerator. Reference tokens are chosen for meaningful trade + volume in the measurement window so a coverage gap actually shows. + Providers use their official public endpoints. Mobula through + /api/2/trades/filters, Bitquery through streaming.bitquery.io + GraphQL, Codex through graph.codex.io GraphQL. + Stellar is measured only for providers that officially list a + Stellar trades endpoint; providers that do not index Stellar at all + are omitted from that chain's row rather than counted as zero. + +methodology: + - "Reference tokens: a fixed set per chain, chosen for enough trade volume in the measurement window that a coverage gap is visible above sampling noise." + - "Measurement window: 60 minutes rolling. The window is the same for every provider in a given cycle so counts are directly comparable." + - "Mobula: `GET https://api.mobula.io/api/2/trades/filters?from={ts_ms}&to={ts_ms}` with `Authorization: `. Cursor pagination, limit=5000." + - "Bitquery: `POST https://streaming.bitquery.io/graphql` with `X-API-KEY`. GraphQL query on `DEXTradeByTokens` with `Block.Time.since` / `till` on the realtime dataset. Hard limit 10000 per call." + - "Codex: `POST https://graph.codex.io/graphql` with `Authorization: `. GraphQL query on `getTokenEvents` with `timestamp` bounds. Cursor pagination." + - "Stellar coverage measured only for providers that ship an official Stellar trades endpoint. Providers with no Stellar support are excluded from the Stellar row rather than counted as zero (that would penalise all three providers unfairly against a chain none of them promised)." + - "Capture rate: `provider_count / max(provider_count across all providers on that (chain, token)) * 100`. Union baseline is the largest observed count, not a synthetic ground truth." + - "Cadence: every 30 minutes per (provider, chain, token). One measurement is short (a handful of API calls per token) so the full cycle finishes well inside the 30-minute window." + - "Failure handling: HTTP timeouts and 5xx count as zero trades for that cycle. Persistent 5xx (three cycles in a row) surfaces as `unresponsive` on the leaderboard, not silent zeros." + +findings: + - "{{best_name}} currently leads at {{best_p50}} average trade capture across {{count}} measured providers. The union baseline updates every cycle: as one provider extends coverage the ceiling moves and the others' relative capture rates shift." + - "{{name:mobula}} averages {{p50:mobula}}. Mobula's advantage on this bench is DEX breadth. Coverage of pump.fun, meme AMMs and small aggregator DEXs is what drives the number up on new-launch Solana tokens where competitors often index only Raydium or Meteora." + - "{{name:bitquery}} averages {{p50:bitquery}}. Bitquery ships strong Solana and EVM coverage through its `DEXTradeByTokens` GraphQL, but the 10000-row hard cap on a single query truncates high-volume tokens in the first hour after launch; the harness paginates when possible, but not every trade type is paginable." + - "{{name:codex}} averages {{p50:codex}}. Codex (formerly Defined) prioritises depth on EVM and Solana DEX event indexing, which shows here as strong per-chain coverage on the chains they index and no coverage at all on Stellar." + - "Stellar coverage is uneven: only providers that officially list a Stellar trades endpoint appear on that chain's row. This is the honest way to represent the current state of the market. Mobula added Stellar recently, others have not shipped it yet." + +source: https://github.com/ChainBench/OpenChainBench/tree/main/harnesses/token-trade-coverage + +prometheus: + window: 24h + expected_freshness_seconds: 3600 + +faq: + - q: "Which crypto data API returns the most complete trade history for a token?" + a: "{{best_name}} currently leads at {{best_p50}} average trade capture across {{count}} measured providers. This is measured per (chain, token) and averaged, so the leaderboard reflects real coverage across a mixed sample rather than a single lucky token." + - q: "What does 'capture rate' actually mean here?" + a: "For each reference token on each chain, the harness fetches trades from every provider over the same time window. The largest count across providers is the baseline (union of what's discoverable through public APIs). Each provider's rate is its own count divided by that baseline, expressed as a percent. A rate of 100 on a token means the provider returned as many trades as the best-covering provider. A rate of 40 means it missed 60 percent of the swaps the best-covering provider found." + - q: "Why not use a canonical onchain ground truth from RPC nodes?" + a: "In principle you could walk the block explorer and reconstruct every swap. In practice the ground truth depends on which DEX programs, protocols and vault contracts you include, and that decision is itself opinionated. The union-of-providers baseline is a defensible proxy that mirrors what a trader integrating a public API can actually see. It undercounts the truth uniformly across providers, so relative ranking is preserved even if absolute completeness is not 100." + - q: "Which chains are covered?" + a: "Solana, Ethereum, BSC, Base and Stellar. Every provider is measured on each chain it officially supports. Stellar is included because Mobula added Stellar trades to its API recently; other providers that do not ship a Stellar endpoint are excluded from the Stellar row rather than counted as zero." + +dimensions: + chain: + - value: all + label: All chains + - value: solana + label: Solana + - value: ethereum + label: Ethereum + - value: bsc + label: BNB Chain + - value: base + label: Base + - value: stellar + label: Stellar + +providers: + - slug: mobula + name: Mobula + tag: /api/2/trades/filters + formula: "Trades returned by Mobula /api/2/trades/filters divided by union-across-providers count on the same (chain, token) in a rolling 60 min window." + queries: + p50: avg_over_time(ocb_token_trade_capture_pct{provider="mobula"}[24h]) + p90: quantile_over_time(0.90, ocb_token_trade_capture_pct{provider="mobula"}[24h]) + p99: quantile_over_time(0.99, ocb_token_trade_capture_pct{provider="mobula"}[24h]) + mean: avg_over_time(ocb_token_trade_capture_pct{provider="mobula"}[24h]) + success: clamp_max(avg_over_time(ocb_token_trade_probe_ok{provider="mobula"}[24h]), 1) + sample_size: count_over_time(ocb_token_trade_capture_pct{provider="mobula"}[24h]) + series: ocb_token_trade_capture_pct{provider="mobula"} + + - slug: bitquery + name: Bitquery + tag: streaming.bitquery.io GraphQL + formula: "Trades returned by Bitquery DEXTradeByTokens (realtime dataset) divided by union-across-providers count on the same (chain, token) in a rolling 60 min window." + queries: + p50: avg_over_time(ocb_token_trade_capture_pct{provider="bitquery"}[24h]) + p90: quantile_over_time(0.90, ocb_token_trade_capture_pct{provider="bitquery"}[24h]) + p99: quantile_over_time(0.99, ocb_token_trade_capture_pct{provider="bitquery"}[24h]) + mean: avg_over_time(ocb_token_trade_capture_pct{provider="bitquery"}[24h]) + success: clamp_max(avg_over_time(ocb_token_trade_probe_ok{provider="bitquery"}[24h]), 1) + sample_size: count_over_time(ocb_token_trade_capture_pct{provider="bitquery"}[24h]) + series: ocb_token_trade_capture_pct{provider="bitquery"} + + - slug: codex + name: Codex + tag: graph.codex.io GraphQL + formula: "Trades returned by Codex getTokenEvents divided by union-across-providers on the same (chain, token) in a rolling 60 min window. Excluded from Stellar row." + queries: + p50: avg_over_time(ocb_token_trade_capture_pct{provider="codex"}[24h]) + p90: quantile_over_time(0.90, ocb_token_trade_capture_pct{provider="codex"}[24h]) + p99: quantile_over_time(0.99, ocb_token_trade_capture_pct{provider="codex"}[24h]) + mean: avg_over_time(ocb_token_trade_capture_pct{provider="codex"}[24h]) + success: clamp_max(avg_over_time(ocb_token_trade_probe_ok{provider="codex"}[24h]), 1) + sample_size: count_over_time(ocb_token_trade_capture_pct{provider="codex"}[24h]) + series: ocb_token_trade_capture_pct{provider="codex"}