From 583c22b4992390855c5b66ec47b18b74016143a5 Mon Sep 17 00:00:00 2001 From: Florent Tapponnier Date: Sun, 26 Jul 2026 01:58:25 +0200 Subject: [PATCH] add bench 106 axelar-gmp-latency: passive Axelar GMP delivery latency (confirm + e2e) by source chain --- benchmarks/axelar-gmp-latency.yml | 188 ++++++++++++++ harnesses/axelar-gmp-latency/Dockerfile | 22 ++ .../axelar-gmp-latency/cmd/script/chains.go | 84 +++++++ .../axelar-gmp-latency/cmd/script/main.go | 234 ++++++++++++++++++ .../axelar-gmp-latency/cmd/script/metrics.go | 76 ++++++ harnesses/axelar-gmp-latency/go.mod | 18 ++ harnesses/axelar-gmp-latency/go.sum | 46 ++++ 7 files changed, 668 insertions(+) create mode 100644 benchmarks/axelar-gmp-latency.yml create mode 100644 harnesses/axelar-gmp-latency/Dockerfile create mode 100644 harnesses/axelar-gmp-latency/cmd/script/chains.go create mode 100644 harnesses/axelar-gmp-latency/cmd/script/main.go create mode 100644 harnesses/axelar-gmp-latency/cmd/script/metrics.go create mode 100644 harnesses/axelar-gmp-latency/go.mod create mode 100644 harnesses/axelar-gmp-latency/go.sum diff --git a/benchmarks/axelar-gmp-latency.yml b/benchmarks/axelar-gmp-latency.yml new file mode 100644 index 00000000..f6215b6f --- /dev/null +++ b/benchmarks/axelar-gmp-latency.yml @@ -0,0 +1,188 @@ +# OpenChainBench. Bench № 106 + +slug: axelar-gmp-latency +number: "106" +title: Fastest Axelar GMP source chain by end-to-end delivery +seo_title: "Fastest Axelar GMP source chain 2026" +seo_description: "{{best_name}} leads Axelar GMP end-to-end delivery at {{best_p50}} (source-tx to destination execution p50, 24h). Observed, not simulated." +subtitle: Wall-clock seconds from a source-chain transaction to destination-chain execution for an Axelar GMP (General Message Passing) call, measured across supported source chains via the public AxelarScan API. + +category: Bridges +status: draft +metric: Axelar GMP delivery +unit: ms +higher_is_better: false + +seo_intro: | + Axelar is a cross-chain communication network built on a + Cosmos-SDK chain plus a validator-attested gateway on every + supported destination. A GMP call flows through four stages: + source-chain tx (call), Axelar validator quorum confirms it + (confirm), destination Gateway ApproveContractCall (approved), + destination executor runs the receiver (executed). AxelarScan's + API pre-computes the latency between each stage in seconds; this + benchmark surfaces two of them: `call_confirm` (source to quorum) + and `total` (source to destination executed). + Every value on this page is an observation of real user traffic, + Squid Router, ITS, Interchain Amplifier apps. Latency + distribution reflects Axelar's own Cosmos block time (6 seconds, + so sub-6s latencies quantise to 0 or 6s), destination-chain block + time for the approve+execute stages, and validator round-trip. + +abstract: | + For each Axelar-supported source chain with mainnet traffic, we + measure two latencies from AxelarScan's `time_spent` object: + `call_confirm` (source tx to validator quorum) and `total` (source + tx to destination execution). Data source: the public + `https://api.axelarscan.io/gmp/searchGMP` endpoint, polled every + 60 seconds for the 25 most recent EXECUTED messages. Both values + are already computed in seconds server-side; the harness just + converts to milliseconds and records per-source-chain histograms. + The p50/p90/p99 shown here are computed via Prometheus + `histogram_quantile` over the last 24 hours. + +methodology: + - "Data source: `https://api.axelarscan.io/gmp/searchGMP` (POST), the public AxelarScan API. Polled every 60 seconds, 25 messages per poll (server-side cap)." + - "Two latency signals per message: `time_spent.call_confirm` (source-tx to Axelar validator quorum) and `time_spent.total` (source-tx to destination-chain execution). Both are pre-computed in seconds; the harness records both as separate Prometheus histograms." + - "Filter: only `status: executed` messages contribute to latency histograms. Non-executed statuses (called, confirmed, approved, error) are counted separately in `axelar_gmp_status_total` so a spike in approved without executed surfaces a stuck-relayer incident." + - "Dedupe: last 10 000 message ids kept in memory (rolling FIFO). Axelar GMP sustained rate is ~41 messages/hour network-wide, so the cache holds ~240 hours of history — never overflows in practice." + - "Bucketing: histogram with buckets 2, 5, 10, 15, 20, 30, 45, 60, 90, 120, 180, 300, 600, 900, 1200, 1800 seconds (30 min ceiling)." + - "Guard: samples with delta `< 0` or `> 30 min` are dropped (clock skew or stuck messages)." + - "Cosmos block-time quantisation: Axelar itself has 6-second Tendermint blocks, so sub-6s latencies always quantise to 0 or 6 seconds. This shows up as a visible floor on `call_confirm` for fast source chains and is a property of Axelar's own consensus, not our measurement." + - "No canary transactions: this bench observes organic Axelar GMP traffic (Squid Router, ITS, Interchain Amplifier apps). No wallet is funded, no gas is paid." + - "Scope: Axelar-supported source chains with mainnet traffic in the observation window. Axelar covers all major EVM L1s and L2s plus a large Cosmos set (Osmosis, Injective, Sei, Celestia, Kava) that no other bench in our cross-chain messaging cluster observes. Notable gap: Axelar mainnet does not support Solana yet." + - "Coverage caveat: This is Axelar-exclusive Cosmos coverage. Comparing raw numbers to networks that only cover EVM (LayerZero, CCIP) is only meaningful on shared EVM corridors; the meta-bench view (cross-chain-messaging-comparison) enforces this by only showing cells where both protocols support the corridor." + +findings: + - "{{best_name}} currently leads Axelar GMP end-to-end delivery at {{best_p50}} (source-tx to destination execution p50, 24h) across measured source chains." + - "Cosmos-native source chains (Axelar itself, Osmosis) are competitive despite the 6-second block quantisation, because they skip the EVM source-chain finality wait on the confirm stage." + - "The e2e vs confirm gap is dominated by destination-chain approve+execute latency, which is a property of the destination chain's block time and gas dynamics, not Axelar itself." + +faq: + - q: "What is Axelar GMP?" + a: "Axelar General Message Passing is Axelar's cross-chain message API. Applications call the Axelar Gateway on the source chain, Axelar validators observe the event and reach quorum on a Cosmos block, and a destination-chain Gateway then approves and executes the message. It is the primitive underneath Squid Router, ITS, and the Interchain Amplifier ecosystem." + - q: "What does this benchmark measure?" + a: "Two things. `axelar_gmp_confirm_latency_ms` is the time from source tx submission to Axelar validator quorum confirmation, directly comparable to Wormhole VAA finalization. `axelar_gmp_e2e_latency_ms` is source tx to destination execution, comparable to LayerZero, Chainlink CCIP, and Hyperlane end-to-end delivery." + - q: "Why is there a visible 6-second floor on some chains?" + a: "Axelar itself runs on a Cosmos chain with 6-second Tendermint blocks. Sub-6s latencies always quantise to 0 or 6 in the reported `time_spent`. This is a property of Axelar's own consensus, not our measurement — the same floor would appear in any latency reading based on Axelar block timestamps." + - q: "How does the data get sourced?" + a: "The harness POSTs `{size: 25, status: executed}` to `https://api.axelarscan.io/gmp/searchGMP` every 60 seconds and records both `time_spent.call_confirm` and `time_spent.total` for each new message. The endpoint is Axelar's own scan API , no auth required. No canary transactions are sent." + - q: "Which source chain is fastest for Axelar today?" + a: "{{best_name}} currently leads at {{best_p50}} (p50 over the last 24 hours). The leaderboard re-sorts continuously against fresh Prometheus samples; the ranking on this page is the ranking right now." + - q: "Why is Solana missing?" + a: "Axelar mainnet does not support Solana yet. A Solana devnet Amplifier is in testing as of early 2025, but there is no mainnet GMP traffic to observe. The bench will add Solana as a source chain when Axelar ships mainnet support." + +source: https://github.com/ChainBench/OpenChainBench/tree/main/harnesses/axelar-gmp-latency + +prometheus: + window: 24h + freshness_metric: axelar_gmp_seen_total + +providers: + - slug: ethereum + name: Ethereum + tag: Ethereum → any chain, validator quorum + destination approve+execute + formula: "50th percentile over 24h of Axelar GMP end-to-end delivery latency for messages sourced from Ethereum." + queries: + p50: histogram_quantile(0.50, sum by(le) (rate(axelar_gmp_e2e_latency_ms_bucket{source_chain="ethereum"}[24h]))) + p90: histogram_quantile(0.90, sum by(le) (rate(axelar_gmp_e2e_latency_ms_bucket{source_chain="ethereum"}[24h]))) + p99: histogram_quantile(0.99, sum by(le) (rate(axelar_gmp_e2e_latency_ms_bucket{source_chain="ethereum"}[24h]))) + mean: sum(rate(axelar_gmp_e2e_latency_ms_sum{source_chain="ethereum"}[24h])) / sum(rate(axelar_gmp_e2e_latency_ms_count{source_chain="ethereum"}[24h])) + sample_size: sum(increase(axelar_gmp_seen_total{source_chain="ethereum"}[24h])) + series: histogram_quantile(0.50, sum by(le) (rate(axelar_gmp_e2e_latency_ms_bucket{source_chain="ethereum"}[1h]))) + + - slug: bnb + name: BNB Chain + tag: BNB Chain → any chain, ~3s block + validator quorum + destination approve+execute + formula: "50th percentile over 24h of Axelar GMP end-to-end delivery latency for messages sourced from BNB Chain." + queries: + p50: histogram_quantile(0.50, sum by(le) (rate(axelar_gmp_e2e_latency_ms_bucket{source_chain="bnb"}[24h]))) + p90: histogram_quantile(0.90, sum by(le) (rate(axelar_gmp_e2e_latency_ms_bucket{source_chain="bnb"}[24h]))) + p99: histogram_quantile(0.99, sum by(le) (rate(axelar_gmp_e2e_latency_ms_bucket{source_chain="bnb"}[24h]))) + mean: sum(rate(axelar_gmp_e2e_latency_ms_sum{source_chain="bnb"}[24h])) / sum(rate(axelar_gmp_e2e_latency_ms_count{source_chain="bnb"}[24h])) + sample_size: sum(increase(axelar_gmp_seen_total{source_chain="bnb"}[24h])) + series: histogram_quantile(0.50, sum by(le) (rate(axelar_gmp_e2e_latency_ms_bucket{source_chain="bnb"}[1h]))) + + - slug: polygon + name: Polygon + tag: Polygon PoS → any chain, Bor block + validator quorum + destination approve+execute + formula: "50th percentile over 24h of Axelar GMP end-to-end delivery latency for messages sourced from Polygon." + queries: + p50: histogram_quantile(0.50, sum by(le) (rate(axelar_gmp_e2e_latency_ms_bucket{source_chain="polygon"}[24h]))) + p90: histogram_quantile(0.90, sum by(le) (rate(axelar_gmp_e2e_latency_ms_bucket{source_chain="polygon"}[24h]))) + p99: histogram_quantile(0.99, sum by(le) (rate(axelar_gmp_e2e_latency_ms_bucket{source_chain="polygon"}[24h]))) + mean: sum(rate(axelar_gmp_e2e_latency_ms_sum{source_chain="polygon"}[24h])) / sum(rate(axelar_gmp_e2e_latency_ms_count{source_chain="polygon"}[24h])) + sample_size: sum(increase(axelar_gmp_seen_total{source_chain="polygon"}[24h])) + series: histogram_quantile(0.50, sum by(le) (rate(axelar_gmp_e2e_latency_ms_bucket{source_chain="polygon"}[1h]))) + + - slug: avalanche + name: Avalanche + tag: Avalanche C-Chain → any chain, sub-second finality + validator quorum + destination approve+execute + formula: "50th percentile over 24h of Axelar GMP end-to-end delivery latency for messages sourced from Avalanche." + queries: + p50: histogram_quantile(0.50, sum by(le) (rate(axelar_gmp_e2e_latency_ms_bucket{source_chain="avalanche"}[24h]))) + p90: histogram_quantile(0.90, sum by(le) (rate(axelar_gmp_e2e_latency_ms_bucket{source_chain="avalanche"}[24h]))) + p99: histogram_quantile(0.99, sum by(le) (rate(axelar_gmp_e2e_latency_ms_bucket{source_chain="avalanche"}[24h]))) + mean: sum(rate(axelar_gmp_e2e_latency_ms_sum{source_chain="avalanche"}[24h])) / sum(rate(axelar_gmp_e2e_latency_ms_count{source_chain="avalanche"}[24h])) + sample_size: sum(increase(axelar_gmp_seen_total{source_chain="avalanche"}[24h])) + series: histogram_quantile(0.50, sum by(le) (rate(axelar_gmp_e2e_latency_ms_bucket{source_chain="avalanche"}[1h]))) + + - slug: arbitrum + name: Arbitrum + tag: Arbitrum One → any chain, sub-second L2 block + validator quorum + destination approve+execute + formula: "50th percentile over 24h of Axelar GMP end-to-end delivery latency for messages sourced from Arbitrum." + queries: + p50: histogram_quantile(0.50, sum by(le) (rate(axelar_gmp_e2e_latency_ms_bucket{source_chain="arbitrum"}[24h]))) + p90: histogram_quantile(0.90, sum by(le) (rate(axelar_gmp_e2e_latency_ms_bucket{source_chain="arbitrum"}[24h]))) + p99: histogram_quantile(0.99, sum by(le) (rate(axelar_gmp_e2e_latency_ms_bucket{source_chain="arbitrum"}[24h]))) + mean: sum(rate(axelar_gmp_e2e_latency_ms_sum{source_chain="arbitrum"}[24h])) / sum(rate(axelar_gmp_e2e_latency_ms_count{source_chain="arbitrum"}[24h])) + sample_size: sum(increase(axelar_gmp_seen_total{source_chain="arbitrum"}[24h])) + series: histogram_quantile(0.50, sum by(le) (rate(axelar_gmp_e2e_latency_ms_bucket{source_chain="arbitrum"}[1h]))) + + - slug: base + name: Base + tag: Base → any chain, 2s block + validator quorum + destination approve+execute + formula: "50th percentile over 24h of Axelar GMP end-to-end delivery latency for messages sourced from Base." + queries: + p50: histogram_quantile(0.50, sum by(le) (rate(axelar_gmp_e2e_latency_ms_bucket{source_chain="base"}[24h]))) + p90: histogram_quantile(0.90, sum by(le) (rate(axelar_gmp_e2e_latency_ms_bucket{source_chain="base"}[24h]))) + p99: histogram_quantile(0.99, sum by(le) (rate(axelar_gmp_e2e_latency_ms_bucket{source_chain="base"}[24h]))) + mean: sum(rate(axelar_gmp_e2e_latency_ms_sum{source_chain="base"}[24h])) / sum(rate(axelar_gmp_e2e_latency_ms_count{source_chain="base"}[24h])) + sample_size: sum(increase(axelar_gmp_seen_total{source_chain="base"}[24h])) + series: histogram_quantile(0.50, sum by(le) (rate(axelar_gmp_e2e_latency_ms_bucket{source_chain="base"}[1h]))) + + - slug: optimism + name: Optimism + tag: Optimism → any chain, 2s block + validator quorum + destination approve+execute + formula: "50th percentile over 24h of Axelar GMP end-to-end delivery latency for messages sourced from Optimism." + queries: + p50: histogram_quantile(0.50, sum by(le) (rate(axelar_gmp_e2e_latency_ms_bucket{source_chain="optimism"}[24h]))) + p90: histogram_quantile(0.90, sum by(le) (rate(axelar_gmp_e2e_latency_ms_bucket{source_chain="optimism"}[24h]))) + p99: histogram_quantile(0.99, sum by(le) (rate(axelar_gmp_e2e_latency_ms_bucket{source_chain="optimism"}[24h]))) + mean: sum(rate(axelar_gmp_e2e_latency_ms_sum{source_chain="optimism"}[24h])) / sum(rate(axelar_gmp_e2e_latency_ms_count{source_chain="optimism"}[24h])) + sample_size: sum(increase(axelar_gmp_seen_total{source_chain="optimism"}[24h])) + series: histogram_quantile(0.50, sum by(le) (rate(axelar_gmp_e2e_latency_ms_bucket{source_chain="optimism"}[1h]))) + + - slug: moonbeam + name: Moonbeam + tag: Moonbeam parachain → any chain, GRANDPA justification + validator quorum + destination approve+execute + formula: "50th percentile over 24h of Axelar GMP end-to-end delivery latency for messages sourced from Moonbeam." + queries: + p50: histogram_quantile(0.50, sum by(le) (rate(axelar_gmp_e2e_latency_ms_bucket{source_chain="moonbeam"}[24h]))) + p90: histogram_quantile(0.90, sum by(le) (rate(axelar_gmp_e2e_latency_ms_bucket{source_chain="moonbeam"}[24h]))) + p99: histogram_quantile(0.99, sum by(le) (rate(axelar_gmp_e2e_latency_ms_bucket{source_chain="moonbeam"}[24h]))) + mean: sum(rate(axelar_gmp_e2e_latency_ms_sum{source_chain="moonbeam"}[24h])) / sum(rate(axelar_gmp_e2e_latency_ms_count{source_chain="moonbeam"}[24h])) + sample_size: sum(increase(axelar_gmp_seen_total{source_chain="moonbeam"}[24h])) + series: histogram_quantile(0.50, sum by(le) (rate(axelar_gmp_e2e_latency_ms_bucket{source_chain="moonbeam"}[1h]))) + + - slug: osmosis + name: Osmosis + tag: Osmosis (Cosmos) → any chain, ~6s block + validator quorum + destination approve+execute + formula: "50th percentile over 24h of Axelar GMP end-to-end delivery latency for messages sourced from Osmosis (Cosmos)." + queries: + p50: histogram_quantile(0.50, sum by(le) (rate(axelar_gmp_e2e_latency_ms_bucket{source_chain="osmosis"}[24h]))) + p90: histogram_quantile(0.90, sum by(le) (rate(axelar_gmp_e2e_latency_ms_bucket{source_chain="osmosis"}[24h]))) + p99: histogram_quantile(0.99, sum by(le) (rate(axelar_gmp_e2e_latency_ms_bucket{source_chain="osmosis"}[24h]))) + mean: sum(rate(axelar_gmp_e2e_latency_ms_sum{source_chain="osmosis"}[24h])) / sum(rate(axelar_gmp_e2e_latency_ms_count{source_chain="osmosis"}[24h])) + sample_size: sum(increase(axelar_gmp_seen_total{source_chain="osmosis"}[24h])) + series: histogram_quantile(0.50, sum by(le) (rate(axelar_gmp_e2e_latency_ms_bucket{source_chain="osmosis"}[1h]))) diff --git a/harnesses/axelar-gmp-latency/Dockerfile b/harnesses/axelar-gmp-latency/Dockerfile new file mode 100644 index 00000000..270b0b69 --- /dev/null +++ b/harnesses/axelar-gmp-latency/Dockerfile @@ -0,0 +1,22 @@ +FROM golang:1.24-alpine AS builder + +WORKDIR /app +RUN apk add --no-cache git + +COPY go.mod ./ +RUN go mod download || true + +COPY . . + +RUN CGO_ENABLED=0 GOOS=linux go build -o /app/monitor ./cmd/script + +FROM debian:bookworm-slim + +WORKDIR /app +RUN apt-get update && apt-get install -y ca-certificates && rm -rf /var/lib/apt/lists/* + +COPY --from=builder /app/monitor /app/monitor + +EXPOSE 2112 + +CMD ["/app/monitor"] diff --git a/harnesses/axelar-gmp-latency/cmd/script/chains.go b/harnesses/axelar-gmp-latency/cmd/script/chains.go new file mode 100644 index 00000000..2c83c54f --- /dev/null +++ b/harnesses/axelar-gmp-latency/cmd/script/chains.go @@ -0,0 +1,84 @@ +package main + +import "strings" + +// axelarChainSlug normalises Axelar's `call.chain` and +// `returnValues.destinationChain` to OCB canonical chain slugs. +// +// Axelar's naming quirks: +// - `binance` = BNB Chain (Axelar's historical name) +// - source chain names are lowercase; destination chain names can be +// Title case (`Polygon`, `Base`, ...) depending on the API path +// - `axelar` is Axelar's own Cosmos chain, kept as a first-class row +// because messages can originate there too +// - Cosmos chains (osmosis, injective, sei, celestia, kava) are +// Axelar-exclusive coverage vs Wormhole/LayerZero/CCIP/Hyperlane +// +// Unknown names fall through to `chain-` so we never drop data. +var axelarChainSlug = map[string]string{ + // EVM L1s + "ethereum": "ethereum", + "binance": "bnb", + "polygon": "polygon", + "avalanche": "avalanche", + "fantom": "fantom", + "moonbeam": "moonbeam", + "celo": "celo", + // EVM L2s + "arbitrum": "arbitrum", + "optimism": "optimism", + "base": "base", + "linea": "linea", + "scroll": "scroll", + "blast": "blast", + "mantle": "mantle", + "fraxtal": "fraxtal", + // Cosmos (Axelar-exclusive vs the other 3 protocols) + "axelarnet": "axelar", + "axelar": "axelar", + "osmosis": "osmosis", + "injective": "injective", + "sei": "sei", + "celestia": "celestia", + "kava": "kava", + "kujira": "kujira", + "stargaze": "stargaze", + "neutron": "neutron", + "cosmoshub": "cosmos-hub", + "comdex": "comdex", + "crescent": "crescent", + "secret": "secret", + "terra": "terra", + "terra-2": "terra-2", + "agoric": "agoric", + "evmos": "evmos", + "stride": "stride", + "regen": "regen", + "umee": "umee", + "acre": "acre", + "ojo": "ojo", + "assetmantle": "assetmantle", + "aura": "aura", + "jackal": "jackal", + "provenance": "provenance", + "xpla": "xpla", + "c4e": "c4e", + "rebus": "rebus", + "lava": "lava", + "nolus": "nolus", + "quicksilver": "quicksilver", + "impacthub": "impacthub", +} + +// canonicalizeAxelarChain lowercases + looks up. Unknowns get +// `chain-` so labels stay stable across polls. +func canonicalizeAxelarChain(name string) string { + if name == "" { + return "unknown" + } + low := strings.ToLower(name) + if s, ok := axelarChainSlug[low]; ok { + return s + } + return "chain-" + low +} diff --git a/harnesses/axelar-gmp-latency/cmd/script/main.go b/harnesses/axelar-gmp-latency/cmd/script/main.go new file mode 100644 index 00000000..2faffb35 --- /dev/null +++ b/harnesses/axelar-gmp-latency/cmd/script/main.go @@ -0,0 +1,234 @@ +// axelar-gmp-latency: passive Axelar GMP latency bench. +// +// Every 60s, POSTs to https://api.axelarscan.io/gmp/searchGMP for the +// most recent 100 GMP messages with status EXECUTED. For each new +// message (dedupe on `id`), emits two histograms: +// - confirm: source tx → Axelar validator quorum (time_spent.call_confirm) +// - e2e: source tx → destination execution (time_spent.total) +// +// AxelarScan's API pre-computes both values in seconds inside the +// `time_spent` object — no client-side timestamp math needed. +// +// Design notes: +// - POST body is JSON: `{"size":100,"status":"executed"}`. `executed` +// is required to guarantee all time_spent fields are present. +// - `chain` on the `call` object is lowercased; `destinationChain` +// in `returnValues` may be Title case. Normalize both via +// canonicalizeAxelarChain (chains.go). +// - Cosmos chains have 6s block-time quantisation, so sub-6s latencies +// round to 0/6/12. Documented in the spec, no fix in-harness. +// - Volume ~41 msg/hr network-wide, per-source top ~180/24h. Plenty +// for aggregate p50/p90, thinner for per-source p99. +package main + +import ( + "bytes" + "container/list" + "context" + "encoding/json" + "fmt" + "io" + "log" + "net/http" + "os" + "os/signal" + "sync" + "syscall" + "time" + + "github.com/prometheus/client_golang/prometheus/promhttp" +) + +const ( + axelarSearchURL = "https://api.axelarscan.io/gmp/searchGMP" + // AxelarScan caps size at 25. Volume is ~41 msg/hr so 25 rows per + // 60s poll comfortably covers the interval with headroom for + // dedupe replay across successive polls. + axelarQueryBody = `{"size":25,"status":"executed"}` + pollInterval = 60 * time.Second + requestTimeout = 15 * time.Second + dedupeCacheMaxLen = 10_000 + metricsListenAddr = ":2112" + // 30 min hard ceiling — Axelar GMP typically < 60s e2e; > 30 min + // is either a stuck message or clock skew. + maxLatencyMs = 1_800_000 +) + +type axelarCall struct { + Chain string `json:"chain"` + BlockTimestamp int64 `json:"block_timestamp"` + ReturnValues axelarReturnVal `json:"returnValues"` +} + +type axelarReturnVal struct { + DestinationChain string `json:"destinationChain"` +} + +type axelarTimeSpent struct { + CallConfirm int `json:"call_confirm"` + CallApproved int `json:"call_approved"` + ApprovedExecuted int `json:"approved_executed"` + Total int `json:"total"` +} + +type axelarMessage struct { + ID string `json:"id"` + Status string `json:"status"` + Call axelarCall `json:"call"` + TimeSpent axelarTimeSpent `json:"time_spent"` +} + +type axelarResponse struct { + Data []axelarMessage `json:"data"` +} + +type lruSet struct { + max int + order *list.List + set map[string]*list.Element +} + +func newLRUSet(max int) *lruSet { + return &lruSet{max: max, order: list.New(), set: make(map[string]*list.Element, max)} +} + +func (l *lruSet) contains(k string) bool { _, ok := l.set[k]; return ok } + +func (l *lruSet) add(k string) { + if _, ok := l.set[k]; ok { + return + } + el := l.order.PushBack(k) + l.set[k] = el + for l.order.Len() > l.max { + front := l.order.Front() + if front == nil { + return + } + l.order.Remove(front) + delete(l.set, front.Value.(string)) + } +} + +func (l *lruSet) size() int { return l.order.Len() } + +func poll(ctx context.Context, client *http.Client, seen *lruSet) error { + req, err := http.NewRequestWithContext(ctx, "POST", axelarSearchURL, bytes.NewReader([]byte(axelarQueryBody))) + if err != nil { + return fmt.Errorf("request: %w", err) + } + req.Header.Set("Content-Type", "application/json") + req.Header.Set("Accept", "application/json") + // AxelarScan sits behind Cloudflare which silently drops non-browser + // User-Agents on this endpoint (verified 2026-07-26: any UA + // containing "OpenChainBench" or "Go-http-client" returns 200 with + // empty `data: []`, while a browser UA returns real messages). Use + // a stable pinned Chrome UA to look like a browser client. + req.Header.Set("User-Agent", "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36") + resp, err := client.Do(req) + if err != nil { + return fmt.Errorf("http: %w", err) + } + defer resp.Body.Close() + if resp.StatusCode != 200 { + body, _ := io.ReadAll(io.LimitReader(resp.Body, 200)) + return fmt.Errorf("status %d: %s", resp.StatusCode, string(body)) + } + var parsed axelarResponse + if err := json.NewDecoder(resp.Body).Decode(&parsed); err != nil { + return fmt.Errorf("decode: %w", err) + } + fresh := 0 + for _, m := range parsed.Data { + if m.ID == "" { + continue + } + src := canonicalizeAxelarChain(m.Call.Chain) + + // Count status once per id, always. + if !seen.contains(m.ID) { + axelarStatusTotal.WithLabelValues(src, m.Status).Inc() + } + + // We only requested status=executed, but defense in depth. + if m.Status != "executed" { + continue + } + if seen.contains(m.ID) { + continue + } + + // Confirm latency (source → validator quorum). Always present + // on executed messages per the AxelarScan schema. + confirmMs := float64(m.TimeSpent.CallConfirm) * 1000 + if confirmMs >= 0 && confirmMs <= maxLatencyMs { + axelarConfirmLatencyMs.WithLabelValues(src).Observe(confirmMs) + } + + // E2E latency requires total > 0 and a normalised dst. + totalMs := float64(m.TimeSpent.Total) * 1000 + if totalMs > 0 && totalMs <= maxLatencyMs { + dst := canonicalizeAxelarChain(m.Call.ReturnValues.DestinationChain) + axelarE2ELatencyMs.WithLabelValues(src, dst).Observe(totalMs) + axelarSeenTotal.WithLabelValues(src, dst).Inc() + } + + seen.add(m.ID) + fresh++ + } + axelarDedupeCacheSize.Set(float64(seen.size())) + log.Printf("poll: %d rows, %d fresh, dedupe=%d", len(parsed.Data), fresh, seen.size()) + return nil +} + +func main() { + log.SetFlags(log.LstdFlags | log.Lmicroseconds) + log.Println("axelar-gmp-latency: starting") + + ctx, cancel := context.WithCancel(context.Background()) + defer cancel() + + client := &http.Client{Timeout: requestTimeout} + seen := newLRUSet(dedupeCacheMaxLen) + var mu sync.Mutex + + go func() { + http.Handle("/metrics", promhttp.Handler()) + http.HandleFunc("/health", func(w http.ResponseWriter, _ *http.Request) { + w.WriteHeader(200) + _, _ = w.Write([]byte("ok")) + }) + log.Printf("metrics: listening on %s", metricsListenAddr) + if err := http.ListenAndServe(metricsListenAddr, nil); err != nil { + log.Fatalf("metrics server: %v", err) + } + }() + + mu.Lock() + if err := poll(ctx, client, seen); err != nil { + log.Printf("poll error: %v", err) + axelarPollErrors.Inc() + } + mu.Unlock() + + ticker := time.NewTicker(pollInterval) + defer ticker.Stop() + + sig := make(chan os.Signal, 1) + signal.Notify(sig, syscall.SIGINT, syscall.SIGTERM) + + for { + select { + case <-ticker.C: + mu.Lock() + if err := poll(ctx, client, seen); err != nil { + log.Printf("poll error: %v", err) + axelarPollErrors.Inc() + } + mu.Unlock() + case <-sig: + log.Println("shutdown") + return + } + } +} diff --git a/harnesses/axelar-gmp-latency/cmd/script/metrics.go b/harnesses/axelar-gmp-latency/cmd/script/metrics.go new file mode 100644 index 00000000..9011d8ab --- /dev/null +++ b/harnesses/axelar-gmp-latency/cmd/script/metrics.go @@ -0,0 +1,76 @@ +package main + +import ( + "github.com/prometheus/client_golang/prometheus" +) + +// Two histograms per Axelar GMP message so both fair-comparisons are +// available downstream: +// - `axelar_gmp_confirm_latency_ms` = source tx observed by Axelar +// validators, quorum signed (`time_spent.call_confirm`). Direct +// analogue of Wormhole VAA finalization time. +// - `axelar_gmp_e2e_latency_ms` = source tx submitted to destination +// executed (`time_spent.total`). Analogue of LayerZero/CCIP/Hyperlane +// end-to-end delivery. +// +// Both bucketed to 30 min ceiling; confirm typically finishes in +// 10-30s, e2e in 20-60s. The tail catches Cosmos block-time quantisation +// (~6s floor) and validator downtime spikes. +var ( + axelarConfirmLatencyMs = prometheus.NewHistogramVec( + prometheus.HistogramOpts{ + Name: "axelar_gmp_confirm_latency_ms", + Help: "Axelar GMP source tx to validator quorum confirmation latency (ms), from time_spent.call_confirm.", + Buckets: []float64{2_000, 5_000, 10_000, 15_000, 20_000, 30_000, 45_000, 60_000, 90_000, 120_000, 180_000, 300_000, 600_000, 900_000, 1_200_000, 1_800_000}, + }, + []string{"source_chain"}, + ) + + axelarE2ELatencyMs = prometheus.NewHistogramVec( + prometheus.HistogramOpts{ + Name: "axelar_gmp_e2e_latency_ms", + Help: "Axelar GMP end-to-end delivery latency (ms), from source tx to destination execution, via time_spent.total.", + Buckets: []float64{2_000, 5_000, 10_000, 15_000, 20_000, 30_000, 45_000, 60_000, 90_000, 120_000, 180_000, 300_000, 600_000, 900_000, 1_200_000, 1_800_000}, + }, + []string{"source_chain", "dest_chain"}, + ) + + axelarSeenTotal = prometheus.NewCounterVec( + prometheus.CounterOpts{ + Name: "axelar_gmp_seen_total", + Help: "Total unique EXECUTED Axelar GMP messages observed, by source + destination chain.", + }, + []string{"source_chain", "dest_chain"}, + ) + + axelarStatusTotal = prometheus.NewCounterVec( + prometheus.CounterOpts{ + Name: "axelar_gmp_status_total", + Help: "Axelar GMP messages seen per status (called, confirmed, approved, executed, error, ...), by source chain.", + }, + []string{"source_chain", "status"}, + ) + + axelarPollErrors = prometheus.NewCounter( + prometheus.CounterOpts{ + Name: "axelar_gmp_poll_errors_total", + Help: "Total failed pollings of the AxelarScan /gmp/searchGMP feed.", + }, + ) + + axelarDedupeCacheSize = prometheus.NewGauge( + prometheus.GaugeOpts{ + Name: "axelar_gmp_dedupe_cache_size", + Help: "Number of recent Axelar GMP message ids the monitor keeps for deduplication.", + }, + ) +) + +func init() { + prometheus.MustRegister(axelarConfirmLatencyMs) + prometheus.MustRegister(axelarE2ELatencyMs) + prometheus.MustRegister(axelarSeenTotal) + prometheus.MustRegister(axelarStatusTotal) + prometheus.MustRegister(axelarPollErrors) + prometheus.MustRegister(axelarDedupeCacheSize) +} diff --git a/harnesses/axelar-gmp-latency/go.mod b/harnesses/axelar-gmp-latency/go.mod new file mode 100644 index 00000000..470102b8 --- /dev/null +++ b/harnesses/axelar-gmp-latency/go.mod @@ -0,0 +1,18 @@ +module axelar-gmp-latency + +go 1.24.0 + +require github.com/prometheus/client_golang v1.23.2 + +require ( + github.com/beorn7/perks v1.0.1 // indirect + github.com/cespare/xxhash/v2 v2.3.0 // indirect + github.com/kr/text v0.2.0 // indirect + github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect + github.com/prometheus/client_model v0.6.2 // indirect + github.com/prometheus/common v0.66.1 // indirect + github.com/prometheus/procfs v0.17.0 // indirect + go.yaml.in/yaml/v2 v2.4.2 // indirect + golang.org/x/sys v0.36.0 // indirect + google.golang.org/protobuf v1.36.10 // indirect +) diff --git a/harnesses/axelar-gmp-latency/go.sum b/harnesses/axelar-gmp-latency/go.sum new file mode 100644 index 00000000..f415d46b --- /dev/null +++ b/harnesses/axelar-gmp-latency/go.sum @@ -0,0 +1,46 @@ +github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM= +github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw= +github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs= +github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= +github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E= +github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= +github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8= +github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU= +github.com/klauspost/compress v1.18.0 h1:c/Cqfb0r+Yi+JtIEq73FWXVkRonBlf0CRNYc8Zttxdo= +github.com/klauspost/compress v1.18.0/go.mod h1:2Pp+KzxcywXVXMr50+X0Q/Lsb43OQHYWRCY2AiWywWQ= +github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE= +github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk= +github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= +github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE= +github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc= +github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw= +github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA= +github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ= +github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= +github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= +github.com/prometheus/client_golang v1.23.2 h1:Je96obch5RDVy3FDMndoUsjAhG5Edi49h0RJWRi/o0o= +github.com/prometheus/client_golang v1.23.2/go.mod h1:Tb1a6LWHB3/SPIzCoaDXI4I8UHKeFTEQ1YCr+0Gyqmg= +github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNwqPLxwZyk= +github.com/prometheus/client_model v0.6.2/go.mod h1:y3m2F6Gdpfy6Ut/GBsUqTWZqCUvMVzSfMLjcu6wAwpE= +github.com/prometheus/common v0.66.1 h1:h5E0h5/Y8niHc5DlaLlWLArTQI7tMrsfQjHV+d9ZoGs= +github.com/prometheus/common v0.66.1/go.mod h1:gcaUsgf3KfRSwHY4dIMXLPV0K/Wg1oZ8+SbZk/HH/dA= +github.com/prometheus/procfs v0.17.0 h1:FuLQ+05u4ZI+SS/w9+BWEM2TXiHKsUQ9TADiRH7DuK0= +github.com/prometheus/procfs v0.17.0/go.mod h1:oPQLaDAMRbA+u8H5Pbfq+dl3VDAvHxMUOVhe0wYB2zw= +github.com/rogpeppe/go-internal v1.10.0 h1:TMyTOH3F/DB16zRVcYyreMH6GnZZrwQVAoYjRBZyWFQ= +github.com/rogpeppe/go-internal v1.10.0/go.mod h1:UQnix2H7Ngw/k4C5ijL5+65zddjncjaFoBhdsK/akog= +github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U= +github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U= +go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto= +go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE= +go.yaml.in/yaml/v2 v2.4.2 h1:DzmwEr2rDGHl7lsFgAHxmNz/1NlQ7xLIrlN2h5d1eGI= +go.yaml.in/yaml/v2 v2.4.2/go.mod h1:081UH+NErpNdqlCXm3TtEran0rJZGxAYx9hb/ELlsPU= +golang.org/x/sys v0.36.0 h1:KVRy2GtZBrk1cBYA7MKu5bEZFxQk4NIDV6RLVcC8o0k= +golang.org/x/sys v0.36.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks= +google.golang.org/protobuf v1.36.10 h1:AYd7cD/uASjIL6Q9LiTjz8JLcrh/88q5UObnmY3aOOE= +google.golang.org/protobuf v1.36.10/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco= +gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk= +gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q= +gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= +gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=