diff --git a/benchmarks/axelar-gmp-latency.yml b/benchmarks/axelar-gmp-latency.yml index d95c2628..f8de3a2b 100644 --- a/benchmarks/axelar-gmp-latency.yml +++ b/benchmarks/axelar-gmp-latency.yml @@ -53,6 +53,7 @@ methodology: - "Cosmos block-time quantisation: Axelar itself has 6-second Tendermint blocks, so sub-6s latencies always quantise to 0 or 6 seconds. This shows up as a visible floor on `call_confirm` for fast source chains and is a property of Axelar's own consensus, not our measurement." - "No canary transactions: this bench observes organic Axelar GMP traffic (Squid Router, ITS, Interchain Amplifier apps). No wallet is funded, no gas is paid." - "Scope: Axelar-supported source chains with mainnet traffic in the observation window. Axelar covers all major EVM L1s and L2s plus a large Cosmos set (Osmosis, Injective, Sei, Celestia, Kava) that no other bench in our cross-chain messaging cluster observes. Notable gap: Axelar mainnet does not support Solana yet." + - "Fair-comparison caveat: Axelar validators wait for source-chain confirmations before signing (about 96 blocks / 20 minutes on Ethereum), so Ethereum-source lanes are structurally slower than pre-finality signers (LayerZero, Hyperlane) but still faster than Chainlink CCIP which waits for full Casper FFG finality. The meta-bench (cross-chain-messaging-latency) documents this ranking." - "Coverage caveat: This is Axelar-exclusive Cosmos coverage. Comparing raw numbers to networks that only cover EVM (LayerZero, CCIP) is only meaningful on shared EVM corridors; the meta-bench view (cross-chain-messaging-comparison) enforces this by only showing cells where both protocols support the corridor." findings: diff --git a/benchmarks/cross-chain-messaging-latency.yml b/benchmarks/cross-chain-messaging-latency.yml index 2cd9ccb7..f8cf4e86 100644 --- a/benchmarks/cross-chain-messaging-latency.yml +++ b/benchmarks/cross-chain-messaging-latency.yml @@ -48,7 +48,7 @@ abstract: | methodology: - "Data sources: each protocol's own public message tracker, polled every 60 seconds. LayerZero via scan.layerzero-api.com/v1/messages/latest, Chainlink CCIP via api.ccip.chain.link/v2/messages, Hyperlane via api.hyperlane.xyz/v1/graphql, Axelar via api.axelarscan.io/gmp/searchGMP." - "Latency definition: wall-clock seconds from source-chain transaction submission to destination-chain execution, computed from each tracker's own source and destination block timestamps. All four protocols expose the same shape (source_ts, destination_ts); the harness converts to milliseconds and records a per-source-chain histogram." - - "Fairness caveat: Chainlink CCIP waits for source-chain finality (12-15 min on Ethereum) before its DON commits, so ETH-source lanes look structurally slower on CCIP for security reasons, not performance. The other three (LayerZero, Hyperlane, Axelar) sign pre-finality by default." + - "Fairness caveat on Ethereum-source lanes: CCIP AND Axelar both wait for source-chain finality before signing (CCIP via Casper FFG, Axelar via ~96 confirmations) so ETH-source lanes structurally add 12-20 min. LayerZero and Hyperlane sign pre-finality by default. L2 and non-EVM lanes are more directly comparable because they finalize quickly." - "Wormhole excluded from this side-by-side: wormholescan only exposes source-tx-to-Guardian-quorum (signing) latency, not destination-side execution. Including Wormhole would compare signing time (about 18s) against end-to-end delivery time (about 30-55s) and make Wormhole look 3x faster than it actually is. Wormhole gets its own dedicated bench at wormhole-vaa-latency with the same signing-time methodology." - "Aggregation: the number on this page is the p50/p90/p99 across ALL corridors each protocol observes over the last 24 hours. Per-corridor breakdown is on each dedicated bench (layerzero-message-latency, chainlink-ccip-latency, hyperlane-message-latency, axelar-gmp-latency)." - "Coverage: LayerZero covers 30+ chains including Solana. Chainlink CCIP and Hyperlane cover most EVMs; CCIP added Solana Oct 2025. Axelar covers all major EVMs plus a Cosmos set (Osmosis, Injective, Sei, Celestia) but not Solana. Aggregation is honest because each protocol is aggregated over its own footprint; per-corridor drill-down lives on each dedicated bench." @@ -57,14 +57,14 @@ methodology: findings: - "{{best_name}} currently leads end-to-end cross-chain delivery at {{best_p50}} (p50 aggregate across all corridors, 24h) across the four measured protocols." - - "Pre-finality signing (LayerZero, Hyperlane, Axelar) delivers in the seconds-to-tens-of-seconds range. Chainlink CCIP sits in the minutes range because it waits for source-chain finality before its DON commits. That is a security tradeoff visible directly in the data." + - "Pre-finality signers (LayerZero, Hyperlane) deliver in the seconds-to-tens-of-seconds range. Chainlink CCIP and Axelar both add source-chain finality wait on Ethereum-source lanes (CCIP waits for Casper FFG epoch, Axelar waits for ~96 confirmations), so their aggregate p50 sits higher when Ethereum-source volume dominates their footprint. That is a security tradeoff visible directly in the data." - "Per-corridor variance is large. The aggregate here is a headline; the individual bench for each protocol shows the per-source-chain breakdown that answers what you would feel in production." faq: - q: "Why isn't Wormhole in this comparison?" a: "Wormhole's public tracker (wormholescan) only exposes signing time, source-tx-to-Guardian-quorum. It does not consistently expose destination-side execution timestamps. Including Wormhole here would compare its signing time (about 18s aggregate) against LayerZero/CCIP/Hyperlane/Axelar end-to-end delivery (about 30s-20min) and make Wormhole look 3x faster than it actually is at delivery. Wormhole gets its own dedicated bench at wormhole-vaa-latency with correct signing-only methodology and full disclosure." - - q: "Why is Chainlink CCIP so much slower?" - a: "CCIP intentionally waits for source-chain finality before its DON commits a merkle root. On Ethereum, that means Casper FFG finalization (12-15 minutes typical, up to 25 minutes in adverse epochs). LayerZero, Hyperlane and Axelar all sign pre-finality by default, so their end-to-end delivery excludes that wait. This is a security tradeoff, not a performance bug." + - q: "Why is Chainlink CCIP so much slower on Ethereum-source?" + a: "CCIP waits for source-chain finality before its DON commits a merkle root. On Ethereum, that means Casper FFG finalization (12-15 minutes typical, up to 25 minutes in adverse epochs). LayerZero and Hyperlane both sign pre-finality by default, so their end-to-end delivery excludes that wait. Axelar validators DO wait for source-chain confirmations on Ethereum (about 96 blocks, roughly 20 minutes), so Axelar shows a similar ETH-source slowdown, though smaller than CCIP because Axelar counts source blocks rather than Casper FFG epochs." - q: "Where is Circle CCTP?" a: "Circle's Iris API does not expose a list-recent endpoint, so passive polling requires maintaining nonce state via on-chain event scanning on all ~10 CCTP source chains plus Solana. That is on the roadmap but structurally more expensive than the four protocols measured here (5-7 days of work vs 1-2 days each)." - q: "Is comparing these four raw numbers actually fair?" diff --git a/benchmarks/layerzero-message-latency.yml b/benchmarks/layerzero-message-latency.yml index 5d3e16b7..6f9d8b0b 100644 --- a/benchmarks/layerzero-message-latency.yml +++ b/benchmarks/layerzero-message-latency.yml @@ -54,7 +54,7 @@ methodology: - "Guard: samples with delta `< 0` or `> 30 min` are dropped (clock skew, stuck messages re-executed later)." - "No canary transactions: this bench observes organic LayerZero traffic (~250 messages/hour network-wide). No wallet is funded, no gas is paid." - "Scope: LayerZero-supported source chains with mainnet traffic in the observation window. LayerZero exposes many exotic chains (orderly, flare, ape, robinhood) that show up as separate rows once volume is non-trivial; low-traffic sources will have wide error bars." - - "Fair-comparison caveat: LayerZero DVNs sign pre-finality by default, so end-to-end delivery excludes the source-chain finality wait that finality-aware networks (Chainlink CCIP) impose. Comparing raw numbers to CCIP on ETH-source lanes is not apples-to-apples; the meta-bench view (cross-chain-messaging-comparison) exposes a finality-normalized column." + - "Fair-comparison caveat: LayerZero DVNs sign pre-finality by default, so end-to-end delivery excludes the source-chain finality wait that finality-aware networks (Chainlink CCIP, Axelar on Ethereum-source) impose. Comparing raw numbers to those on ETH-source lanes is not apples-to-apples; the meta-bench (cross-chain-messaging-latency) documents the tradeoff." findings: - "{{best_name}} currently leads LayerZero end-to-end delivery at {{best_p50}} (source-tx to destination execution p50, 24h) across measured source chains." diff --git a/harnesses/axelar-gmp-latency/cmd/script/main.go b/harnesses/axelar-gmp-latency/cmd/script/main.go index 2faffb35..a3652e99 100644 --- a/harnesses/axelar-gmp-latency/cmd/script/main.go +++ b/harnesses/axelar-gmp-latency/cmd/script/main.go @@ -49,9 +49,13 @@ const ( requestTimeout = 15 * time.Second dedupeCacheMaxLen = 10_000 metricsListenAddr = ":2112" - // 30 min hard ceiling — Axelar GMP typically < 60s e2e; > 30 min - // is either a stuck message or clock skew. - maxLatencyMs = 1_800_000 + // 2h hard ceiling. Aligned with the other e2e cross-chain messaging + // benches (CCIP, LayerZero, Hyperlane) so the drop threshold is + // consistent across the cross-chain-messaging-latency meta-bench. + // Axelar validators wait for source-chain finality on EVM (Ethereum + // ~20 min), so the tail legitimately reaches 30-60 min on ETH-source + // lanes and must not be trimmed. + maxLatencyMs = 7_200_000 // 2 hours ) type axelarCall struct { diff --git a/harnesses/axelar-gmp-latency/cmd/script/metrics.go b/harnesses/axelar-gmp-latency/cmd/script/metrics.go index 69794222..2c8bfe0b 100644 --- a/harnesses/axelar-gmp-latency/cmd/script/metrics.go +++ b/harnesses/axelar-gmp-latency/cmd/script/metrics.go @@ -13,15 +13,18 @@ import ( // executed (`time_spent.total`). Analogue of LayerZero/CCIP/Hyperlane // end-to-end delivery. // -// Both bucketed to 30 min ceiling; confirm typically finishes in -// 10-30s, e2e in 20-60s. The tail catches Cosmos block-time quantisation -// (~6s floor) and validator downtime spikes. +// Buckets aligned with the other three e2e cross-chain messaging +// benches (CCIP, LayerZero, Hyperlane) up to 90 min so the meta-bench +// (cross-chain-messaging-latency) compares equivalent bucket +// resolution across all four protocols. Axelar validators wait for +// source-chain finality on Ethereum (~20 min), so ETH-source lanes +// legitimately reach 30-40 min end-to-end and must not be capped. var ( axelarConfirmLatencyMs = prometheus.NewHistogramVec( prometheus.HistogramOpts{ Name: "axelar_gmp_confirm_latency_milliseconds", Help: "Axelar GMP source tx to validator quorum confirmation latency (ms), from time_spent.call_confirm.", - Buckets: []float64{2_000, 5_000, 10_000, 15_000, 20_000, 30_000, 45_000, 60_000, 90_000, 120_000, 180_000, 300_000, 600_000, 900_000, 1_200_000, 1_800_000}, + Buckets: []float64{5_000, 15_000, 30_000, 60_000, 120_000, 180_000, 300_000, 600_000, 900_000, 1_200_000, 1_800_000, 2_400_000, 3_000_000, 3_600_000, 4_500_000, 5_400_000}, }, []string{"source_chain"}, ) @@ -30,7 +33,7 @@ var ( prometheus.HistogramOpts{ Name: "axelar_gmp_e2e_latency_milliseconds", Help: "Axelar GMP end-to-end delivery latency (ms), from source tx to destination execution, via time_spent.total.", - Buckets: []float64{2_000, 5_000, 10_000, 15_000, 20_000, 30_000, 45_000, 60_000, 90_000, 120_000, 180_000, 300_000, 600_000, 900_000, 1_200_000, 1_800_000}, + Buckets: []float64{5_000, 15_000, 30_000, 60_000, 120_000, 180_000, 300_000, 600_000, 900_000, 1_200_000, 1_800_000, 2_400_000, 3_000_000, 3_600_000, 4_500_000, 5_400_000}, }, []string{"source_chain", "dest_chain"}, ) diff --git a/harnesses/hyperlane-message-latency/cmd/script/main.go b/harnesses/hyperlane-message-latency/cmd/script/main.go index 83f8eda6..56ff7dfd 100644 --- a/harnesses/hyperlane-message-latency/cmd/script/main.go +++ b/harnesses/hyperlane-message-latency/cmd/script/main.go @@ -49,9 +49,13 @@ const ( requestTimeout = 15 * time.Second dedupeCacheMaxLen = 10_000 metricsListenAddr = ":2112" - // 30 min hard ceiling — Hyperlane relayers are typically < 2 min - // end-to-end; > 30 min is either a stuck message or clock skew. - maxLatencyMs = 1_800_000 + // 2h hard ceiling. Aligned with the other e2e cross-chain messaging + // benches (CCIP, LayerZero, Axelar) so the drop threshold is + // consistent across the cross-chain-messaging-latency meta-bench. + // Hyperlane relayers are typically < 2 min but real stuck deliveries + // have been observed at 30-90 min; we want the tail preserved for + // meaningful p99 comparison. + maxLatencyMs = 7_200_000 // 2 hours ) type hyperlaneMessage struct { diff --git a/harnesses/hyperlane-message-latency/cmd/script/metrics.go b/harnesses/hyperlane-message-latency/cmd/script/metrics.go index 578fe5c8..f008cd2b 100644 --- a/harnesses/hyperlane-message-latency/cmd/script/metrics.go +++ b/harnesses/hyperlane-message-latency/cmd/script/metrics.go @@ -14,13 +14,18 @@ import ( // theoretical protocol floor. // // Emitted in MILLISECONDS to match the site's `unit: ms` display -// convention. Buckets 2s-30min. +// convention. Buckets aligned with the other three e2e cross-chain +// messaging benches (CCIP, LayerZero, Axelar) so the meta-bench +// (cross-chain-messaging-latency) compares equivalent bucket +// resolution. Hyperlane relayers are typically < 2 min but the +// permissionless relayer model produces real 30-90 min stalls on +// under-served corridors, and the tail must be preserved for honest p99. var ( hyperlaneLatencyMs = prometheus.NewHistogramVec( prometheus.HistogramOpts{ Name: "hyperlane_message_latency_milliseconds", Help: "End-to-end delivery latency (ms) for a Hyperlane message, from source send to destination delivery, labeled by source + destination chain.", - Buckets: []float64{2_000, 5_000, 10_000, 15_000, 20_000, 30_000, 45_000, 60_000, 90_000, 120_000, 180_000, 300_000, 600_000, 900_000, 1_200_000, 1_800_000}, + Buckets: []float64{5_000, 15_000, 30_000, 60_000, 120_000, 180_000, 300_000, 600_000, 900_000, 1_200_000, 1_800_000, 2_400_000, 3_000_000, 3_600_000, 4_500_000, 5_400_000}, }, []string{"source_chain", "dest_chain"}, ) diff --git a/harnesses/layerzero-message-latency/cmd/script/main.go b/harnesses/layerzero-message-latency/cmd/script/main.go index 11a1c7ec..5f7a4f6d 100644 --- a/harnesses/layerzero-message-latency/cmd/script/main.go +++ b/harnesses/layerzero-message-latency/cmd/script/main.go @@ -54,10 +54,12 @@ const ( requestTimeout = 15 * time.Second dedupeCacheMaxLen = 10_000 metricsListenAddr = ":2112" - // Guard against clock skew and stuck messages. LayerZero rarely - // exceeds 15 min end-to-end; anything beyond 30 min is almost - // certainly bad data. - maxLatencyMs = 1_800_000 // 30 min + // Guard against clock skew and stuck messages. Aligned with the + // other e2e cross-chain messaging benches (CCIP, Hyperlane, Axelar) + // at 2 hours so the drop threshold is consistent across the + // cross-chain-messaging-latency meta-bench. Any single message + // beyond 2h is almost certainly a stuck/re-delivered artifact. + maxLatencyMs = 7_200_000 // 2 hours ) type lzTx struct { diff --git a/harnesses/layerzero-message-latency/cmd/script/metrics.go b/harnesses/layerzero-message-latency/cmd/script/metrics.go index 734bfbf4..e719e381 100644 --- a/harnesses/layerzero-message-latency/cmd/script/metrics.go +++ b/harnesses/layerzero-message-latency/cmd/script/metrics.go @@ -13,15 +13,19 @@ import ( // + executor delivery), not just protocol signing latency. // // Emitted in MILLISECONDS to match the site's `unit: ms` display -// convention. Buckets extend to 30 min: LayerZero rarely runs longer -// than a few minutes end-to-end (fewer finality waits than CCIP), but -// the tail catches DVN misconfig or destination-chain congestion. +// convention. Buckets aligned with the other three e2e cross-chain +// messaging benches (CCIP, Hyperlane, Axelar) — up to 90 min — so +// the cross-chain-messaging-latency meta-bench compares equivalent +// bucket resolution across all four protocols. LayerZero typically +// runs a few minutes end-to-end but real stuck deliveries at 30-60 +// min have been observed on low-liquidity DVN configurations, and +// the tail must be preserved for honest p99. var ( lzLatencyMs = prometheus.NewHistogramVec( prometheus.HistogramOpts{ Name: "lz_message_latency_milliseconds", Help: "End-to-end delivery latency (ms) for a LayerZero message, from source-chain block to destination-chain block, labeled by source + destination chain.", - Buckets: []float64{2_000, 5_000, 10_000, 15_000, 20_000, 30_000, 45_000, 60_000, 90_000, 120_000, 180_000, 300_000, 600_000, 900_000, 1_200_000, 1_800_000}, + Buckets: []float64{5_000, 15_000, 30_000, 60_000, 120_000, 180_000, 300_000, 600_000, 900_000, 1_200_000, 1_800_000, 2_400_000, 3_000_000, 3_600_000, 4_500_000, 5_400_000}, }, []string{"source_chain", "dest_chain"}, ) diff --git a/paraswap.png b/paraswap.png new file mode 100644 index 00000000..849a29f7 Binary files /dev/null and b/paraswap.png differ