bench vague 2: ship 081-085 (ws latency, oracle freshness, rpc reliability, indexer latency, block builders)

benchmarks/indexer-latency.yml

@@ -0,0 +1,127 @@
+# OpenChainBench. Bench № 084
+
+slug: indexer-latency
+number: "084"
+title: Fastest blockchain indexer. Envio HyperSync, The Graph, Alchemy
+seo_title: "Fastest blockchain indexer 2026: HyperSync vs The Graph vs Alchemy"
+seo_description: "How fast do indexing pipelines make new onchain events queryable? Envio HyperSync, The Graph and Alchemy measured live on organic Ethereum USDC transfers, second by second."
+subtitle: Seconds between a USDC transfer confirming on Ethereum and the moment each indexing pipeline first makes that block's data queryable, measured continuously on organic transfers.
+category: Aggregators
+status: live
+metric: Time to queryable
+unit: s
+higher_is_better: false
+
+seo_intro: |
+  Every dapp backend, analytics product and trading system sits on an
+  indexing pipeline, and in 2026 the choice of that pipeline is a live
+  question again. Dune is sunsetting Sim, teams are re-evaluating
+  their data stack, and Envio markets HyperSync as "10x faster" than
+  alternatives without publishing comparable numbers. This benchmark
+  measures the claim the only way that cannot be gamed. we pick a
+  random, organic USDC transfer from the newest Ethereum block, then
+  poll each indexing pipeline until that block's data is queryable.
+  Because the ground truth is a random real transaction there is
+  nothing a provider could special case, and every measurement is
+  publicly re-verifiable from the tx hash. Cohort. Envio HyperSync
+  (low level log query engine), The Graph (decentralized subgraph
+  network, probed through the canonical Uniswap V3 subgraph) and
+  Alchemy (indexed transfers API), all probed on the identical
+  schedule from the same host. This cohort is indexing
+  infrastructure; wallet data APIs (Zerion, Moralis, Mobula and
+  friends) are raced separately in bench 070, with zero provider
+  overlap between the two.
+
+abstract: |
+  We measure time to queryable for indexing infrastructure. the delay
+  between an organic USDC transfer confirming on Ethereum (T0 = the
+  instant our own keyless reference RPC first shows the containing
+  block) and the first poll at which each provider can return that
+  block's data. One probe event per 5 minutes; each event uses a fresh
+  random transfer from a fresh block. The poll schedule is front
+  loaded (1s to 240s, identical for every provider) and detection is
+  parser free where the API returns the tx hash verbatim. Because the
+  three pipelines expose different query surfaces, the probed entity
+  differs per provider and is disclosed. HyperSync is asked for the
+  exact Transfer log, Alchemy for the exact transfer via its indexed
+  transfers API, The Graph for a block pinned query on the canonical
+  Uniswap V3 subgraph that only succeeds once the subgraph has indexed
+  the reference block. The measured question is identical in all three
+  cases. is block N data queryable at time T? Events not queryable
+  within 240 seconds count as timeouts and feed the reliability
+  column. Per provider monthly quota guards keep the probe inside
+  every free tier.
+
+methodology:
+  - "Ground truth: one probe event per 5 minutes. The harness watches new Ethereum blocks through its own keyless reference RPC (publicnode, never a cohort member's endpoint) and picks one random USDC Transfer log from the newest block. T0 is the instant the harness first observes the containing block; the same T0 is used for every provider."
+  - "Anti-gaming by construction: the probe target is a random organic USDC transfer, different on every event, so no provider can pre-warm a known benchmark entity. Every measurement is re-verifiable by anyone from the public tx hash. No canary contract is deployed; the benchmark piggybacks on the natural high frequency USDC stream at zero onchain cost."
+  - "Poll schedule: each provider is polled at 1, 2, 3, 5, 8, 12, 20, 30, 45, 60, 90, 120, 180 and 240 seconds after T0. Reported lag is the first poll at which block N's data is queryable, an upper bound with resolution equal to the gap between consecutive polls."
+  - "Probed entity per provider, disclosed because the pipelines expose different query surfaces: HyperSync is queried via POST /query for the exact Transfer log in block N; Alchemy via alchemy_getAssetTransfers with a fromBlock=N filter and the USDC contract; The Graph via a block pinned GraphQL query on the canonical Uniswap V3 Ethereum subgraph, which graph-node rejects until the subgraph has indexed block N. The measured question is identical: is block N data queryable at time T."
+  - "The Graph number therefore includes the full subgraph pipeline (firehose, indexer sync, gateway routing) rather than a raw data lookup. That is the honest comparison: it is the path every subgraph consumer actually waits on, but readers should know it is a pipeline measurement, not a single database read."
+  - "Detection is parser free where possible: HyperSync and Alchemy responses are scanned for the tx hash substring, so no provider gains or loses from response schema differences. The Graph probe succeeds when the block pinned query returns data without a block range error."
+  - "Classification: found (lag recorded), timeout (not queryable within 240s), error (provider errored on every poll of the event), quota_paused (monthly guard tripped), disabled (credential not configured). Percentiles are computed from the histogram of found lags over 24h; the timeout rate is published alongside because latency without reliability is a misleading ranking signal."
+  - "Quota fairness: per provider monthly call budgets sized to each free tier with headroom, enforced by a guard that pauses probing at 90 percent. The Graph joins every second event (100k gateway queries/month free plan); HyperSync and Alchemy join every event. Sample sizes per provider are published."
+  - "Excluded from v1 with reasons: Goldsky (requires deploying and paying for your own subgraph, no neutral shared surface to probe), Dune (2500 free credits/month cannot sustain continuous probing), Bitquery (free tier is a one month trial), Moralis (already raced in bench 070 and its free key is saturated there), SubQuery and Ponder (no measurable hosted free tier)."
+  - "Cohort positioning: this bench races indexing infrastructure. Wallet data APIs (Zerion, Moralis, Allium, GoldRush, Mobula) are a different product layer and are raced separately in bench 070 with an identical T0 and poll design; the two cohorts share zero providers by construction."
+  - "Scope: Ethereum mainnet, single probe region. Time to queryable is measured in seconds while cross region network deltas are milliseconds, so multi region probing would add cost without changing the ranking."
+
+findings:
+  - "{{best_name}} currently leads at {{best_p50}} (p50 of found lags, 24h) on organic Ethereum USDC transfers."
+  - "{{name:hypersync}} sits at {{p50:hypersync}}. Envio's marketing claims 10x faster retrieval than alternatives; this page is the first continuously measured public number behind that claim."
+  - "{{name:thegraph}} runs at {{p50:thegraph}} through the canonical Uniswap V3 subgraph. This is a full pipeline measurement (firehose, indexer, gateway), the path every subgraph consumer actually waits on."
+  - "{{name:alchemy}} sits at {{p50:alchemy}} on its indexed transfers API, the same surface portfolio and activity feeds built on Alchemy consume."
+  - "Read the reliability column before the latency one: a pipeline that occasionally never surfaces an event within 240 seconds breaks downstream consumers in a way a slow median does not."
+
+faq:
+  - q: "Which blockchain indexer makes new events queryable fastest?"
+    a: "Per the live leaderboard above: {{best_name}} at {{best_p50}} (p50 over 24h of organic Ethereum USDC transfers). The ranking re-sorts continuously as probe events land every 5 minutes. Check the reliability column too; a pipeline that occasionally never surfaces an event within 240 seconds is worse for most consumers than one that is a second slower on median."
+  - q: "How is indexer latency measured here?"
+    a: "The harness picks a random organic USDC transfer from the newest Ethereum block, records T0 when its own keyless reference RPC first shows the block, then polls each provider on an identical front loaded schedule (1s to 240s) until that block's data is queryable. HyperSync and Alchemy are asked for the exact transfer; The Graph is asked a block pinned query on the canonical Uniswap V3 subgraph that only succeeds once block N is indexed. The full harness is open source and each measurement can be re-verified from the public tx hash."
+  - q: "Is comparing HyperSync, The Graph and Alchemy apples to apples?"
+    a: "Not perfectly, and the asymmetry is disclosed rather than hidden. HyperSync and Alchemy serve raw or lightly transformed data; The Graph serves subgraph entities, so its number includes the whole firehose plus indexer plus gateway pipeline. All three answer the same operational question a builder has: after an event confirms onchain, how long until my query layer can see it. That is the number this page publishes, per pipeline, with the probed entity documented."
+  - q: "Why are Goldsky, Dune, SubQuery and Ponder missing?"
+    a: "Each lacks a neutral, continuously probeable free surface in v1. Goldsky requires deploying your own subgraph, so any probe would measure our deployment, not the platform baseline. Dune's free tier (2500 credits/month) cannot sustain a 5 minute cadence. Bitquery's free tier is a one month trial. SubQuery and Ponder have no measurable hosted free tier. Moralis is already raced in the wallet data cohort of bench 070. Providers change tiers often; the cohort is revisited when a fair probe becomes possible."
+  - q: "Why does indexer latency matter in 2026?"
+    a: "Because the indexing market is consolidating and re-forming at once. Dune is sunsetting Sim, pushing teams to re-evaluate their data stack, while Envio advertises HyperSync as 10x faster with no public comparable numbers. For trading systems, alerting and activity feeds, the seconds between confirmation and queryability are the product. This page is the only continuously measured, provider neutral comparison of that number across indexing infrastructure."
+
+source: https://github.com/ChainBench/OpenChainBench/tree/main/harnesses/indexer-latency
+
+prometheus:
+  window: 24h
+  freshness_metric: idx_health
+  expected_freshness_seconds: 1800
+
+providers:
+  - slug: hypersync
+    name: Envio HyperSync
+    tag: Low level log query engine, claims 10x faster retrieval
+    formula: "Median (p50) over 24h of seconds between our reference RPC first showing an Ethereum block and HyperSync /query returning that block's probe USDC Transfer log, from the shared histogram estimator."
+    queries:
+      p50: histogram_quantile(0.5, sum by (le) (increase(idx_latency_seconds_bucket{provider="hypersync"}[24h])))
+      p90: histogram_quantile(0.9, sum by (le) (increase(idx_latency_seconds_bucket{provider="hypersync"}[24h])))
+      p99: histogram_quantile(0.99, sum by (le) (increase(idx_latency_seconds_bucket{provider="hypersync"}[24h])))
+      success: sum(increase(idx_probe_total{provider="hypersync", result="found"}[24h])) / sum(increase(idx_probe_total{provider="hypersync", result=~"found|timeout"}[24h]))
+      sample_size: sum(increase(idx_probe_total{provider="hypersync", result=~"found|timeout"}[24h]))
+      series: avg_over_time(idx_latency_last_seconds{provider="hypersync"}[1h])
+  - slug: thegraph
+    name: The Graph
+    tag: Decentralized subgraph network, probed via Uniswap V3 subgraph
+    formula: "Median (p50) over 24h of seconds until a block pinned query on the canonical Uniswap V3 subgraph succeeds for the reference block. Includes the full firehose, indexer and gateway pipeline; joins every second event."
+    queries:
+      p50: histogram_quantile(0.5, sum by (le) (increase(idx_latency_seconds_bucket{provider="thegraph"}[24h])))
+      p90: histogram_quantile(0.9, sum by (le) (increase(idx_latency_seconds_bucket{provider="thegraph"}[24h])))
+      p99: histogram_quantile(0.99, sum by (le) (increase(idx_latency_seconds_bucket{provider="thegraph"}[24h])))
+      success: sum(increase(idx_probe_total{provider="thegraph", result="found"}[24h])) / sum(increase(idx_probe_total{provider="thegraph", result=~"found|timeout"}[24h]))
+      sample_size: sum(increase(idx_probe_total{provider="thegraph", result=~"found|timeout"}[24h]))
+      series: avg_over_time(idx_latency_last_seconds{provider="thegraph"}[1h])
+  - slug: alchemy
+    name: Alchemy
+    tag: Indexed transfers API on the Alchemy data platform
+    formula: "Median (p50) over 24h of seconds between our reference RPC first showing a block and alchemy_getAssetTransfers returning that block's probe USDC transfer with a fromBlock filter, from the shared histogram estimator."
+    queries:
+      p50: histogram_quantile(0.5, sum by (le) (increase(idx_latency_seconds_bucket{provider="alchemy"}[24h])))
+      p90: histogram_quantile(0.9, sum by (le) (increase(idx_latency_seconds_bucket{provider="alchemy"}[24h])))
+      p99: histogram_quantile(0.99, sum by (le) (increase(idx_latency_seconds_bucket{provider="alchemy"}[24h])))
+      success: sum(increase(idx_probe_total{provider="alchemy", result="found"}[24h])) / sum(increase(idx_probe_total{provider="alchemy", result=~"found|timeout"}[24h]))
+      sample_size: sum(increase(idx_probe_total{provider="alchemy", result=~"found|timeout"}[24h]))
+      series: avg_over_time(idx_latency_last_seconds{provider="alchemy"}[1h])