diff --git a/benchmarks/chainlink-ccip-latency.yml b/benchmarks/chainlink-ccip-latency.yml new file mode 100644 index 00000000..53573e1b --- /dev/null +++ b/benchmarks/chainlink-ccip-latency.yml @@ -0,0 +1,179 @@ +# OpenChainBench. Bench № 103 + +slug: chainlink-ccip-latency +number: "103" +title: Fastest Chainlink CCIP source chain by end-to-end delivery latency +seo_title: "Fastest Chainlink CCIP source chain 2026" +seo_description: "{{best_name}} leads Chainlink CCIP end-to-end delivery at {{best_p50}} (source-tx to destination execution p50, 24h) across CCIP source chains. Observed, not simulated." +subtitle: Wall-clock seconds from a source-chain transaction to destination-chain execution for a Chainlink CCIP message, measured across CCIP source chains via the public CCIP Tools API. + +category: Bridges +status: draft +metric: CCIP delivery +unit: ms +higher_is_better: false + +seo_intro: | + Chainlink CCIP (Cross-Chain Interoperability Protocol) is a + cross-chain messaging network operated by the Chainlink DON + (Decentralized Oracle Network). A CCIP message flows through three + stages: source transaction submitted, DON commits a merkle root once + source-chain finality is reached, then a destination executor + performs the delivery. This benchmark measures end-to-end delivery + latency by source chain: wall-clock seconds from source-tx timestamp + to destination-tx timestamp, computed for every CCIP message the + public CCIP Tools API returns as `status: SUCCESS`. + Every value on this page is an observation of real user traffic + (Aave, Vault protocols, cross-chain token bridges), never a canary + transaction. Latency distribution is dominated by source-chain + finality: Ethereum-source lanes routinely take 12-15 minutes waiting + for Casper FFG finalization before the DON commits; L2s that inherit + Ethereum finality show similar structural floors; sub-second BFT + chains (BNB, Solana) finalize fast and land end-to-end in + low-single-digit minutes. Choosing a source chain for a CCIP + integration is a choice about that structural finality tax. + +abstract: | + For each of the CCIP-supported source chains with non-trivial + mainnet traffic, we measure the wall-clock latency between a + source-chain transaction submission and destination-chain + execution. Data source: the public `https://api.ccip.chain.link/v2/messages` + endpoint, polled every 60 seconds. Each `SUCCESS` message's + `receiptTimestamp - sendTimestamp` is one observation of end-to-end + latency, recorded in a per-source-chain histogram. The p50/p90/p99 + shown here are computed via Prometheus `histogram_quantile` over the + last 24 hours. + +methodology: + - "Data source: `https://api.ccip.chain.link/v2/messages?limit=100`, the public CCIP Tools API. Polled every 60 seconds." + - "Latency definition: `receiptTimestamp - sendTimestamp` from a CCIP message row with `status: SUCCESS`. `sendTimestamp` is the source-chain tx block time; `receiptTimestamp` is the destination-chain execution block time." + - "Filter: only mainnet lanes are observed. Testnet messages (Ethereum Sepolia, TON Testnet, etc.) are dropped before histogram observation." + - "Filter: only `status: SUCCESS` messages contribute to latency. In-flight statuses (SENT, SOURCE_FINALIZED, COMMITTED) are counted separately in `ccip_message_status_total` so a spike in COMMITTED without matching SUCCESS surfaces a stuck-execution incident." + - "Dedupe: last 10 000 messageIds kept in memory (rolling FIFO). CCIP sustained rate is ~25 messages/minute network-wide, so the cache holds ~7h of history." + - "Bucketing: histogram with buckets extending to 90 minutes (5 400 s). Wider than the Wormhole VAA bench because CCIP structurally waits for source-chain finality before its DON commits , Ethereum-source p99 legitimately reaches 25-30 min in adverse epochs." + - "Guard: samples with `receiptTimestamp - sendTimestamp < 0` or `> 2h` are dropped (clock skew, stuck messages re-executed later, or timestamp typos)." + - "No canary transactions: this bench observes organic CCIP traffic (~500 mainnet messages/day across the top corridors). No wallet is funded, no gas is paid." + - "Scope: CCIP-supported source chains with mainnet traffic in the observation window. Long-tail source chains (0G, ADI, Astar) will show wide error bars or a low sample count." + - "Fair-comparison caveat: CCIP intentionally waits for source-chain finality before its DON commits, while some other cross-chain messaging networks (Wormhole, LayerZero) sign pre-finality. Comparing raw end-to-end numbers on ETH-source lanes is not apples-to-apples with those networks; the meta-bench view (cross-chain-messaging-comparison) exposes a finality-normalized column that isolates the pure DON+execution work." + +findings: + - "{{best_name}} currently leads Chainlink CCIP end-to-end delivery at {{best_p50}} (source-tx to destination execution p50, 24h) across measured source chains." + - "Sub-second BFT chains lead: BNB Chain and Solana finalize source blocks fast enough that DON commit + execution lands in the low-single-digit minutes rather than the 15+ minute range Ethereum inherits." + - "Ethereum-source lanes are structurally slower by ~12-15 minutes than L2s that don't inherit Ethereum finality. This is a property of the CCIP security model (waits for source finality), not a performance bug." + +faq: + - q: "What is Chainlink CCIP?" + a: "CCIP (Cross-Chain Interoperability Protocol) is a cross-chain messaging network operated by the Chainlink DON. It underpins production integrations at Aave, Vault protocols, and multiple cross-chain token bridges. The security model relies on a Risk Management Network (RMN) and finality-aware settlement , the DON waits for source-chain finality before committing a merkle root." + - q: "What does this benchmark measure?" + a: "Wall-clock seconds from a source-chain transaction to destination-chain execution for CCIP messages that reached `status: SUCCESS`. It is a passive measurement of real user traffic , every observation is a CCIP message that a real user or protocol actually sent." + - q: "Why is Ethereum-source so much slower than other chains?" + a: "CCIP intentionally waits for source-chain finality before its DON commits a merkle root. On Ethereum, that means waiting for Casper FFG finalization , typically 12-15 minutes, up to ~25 minutes in adverse epochs. This is a security choice, not a performance bug. Sub-second BFT chains (BNB, Solana) finalize quickly, so their end-to-end latency is dominated by DON work + destination execution rather than finality wait." + - q: "How does the data get sourced?" + a: "The harness polls the public `https://api.ccip.chain.link/v2/messages` endpoint every 60 seconds and records `receiptTimestamp - sendTimestamp` for each new `SUCCESS` message. The endpoint is Chainlink's own CCIP Tools API , no auth required, ~200ms round-trip. No canary transactions are sent, no user funds are used." + - q: "Which source chain is fastest for CCIP today?" + a: "{{best_name}} currently leads at {{best_p50}} (p50 over the last 24 hours). The leaderboard re-sorts continuously against fresh Prometheus samples; the ranking on this page is the ranking right now." + - q: "Can I compare this to the Wormhole VAA bench?" + a: "Only carefully. CCIP measures end-to-end delivery (source tx → destination execution) after waiting for source-chain finality, while the Wormhole VAA bench measures observation → quorum (pre-finality signing). On Ethereum-source lanes CCIP will look 12-15 minutes slower, but that reflects a security choice, not raw protocol speed. The cross-chain messaging comparison meta-bench exposes a finality-normalized column so the pure DON+execution work is directly comparable to Wormhole's signing latency." + +source: https://github.com/ChainBench/OpenChainBench/tree/main/harnesses/chainlink-ccip-latency + +prometheus: + window: 24h + freshness_metric: ccip_message_seen_total + +providers: + - slug: ethereum + name: Ethereum + tag: Ethereum → any chain, DON commits after Casper FFG finality + formula: "50th percentile over 24h of Chainlink CCIP end-to-end delivery latency for messages sourced from Ethereum: wall-clock delta from source-tx timestamp to destination-execution timestamp, observed via the CCIP Tools API." + queries: + p50: histogram_quantile(0.50, sum by(le) (rate(ccip_message_latency_milliseconds_bucket{source_chain="ethereum"}[24h]))) + p90: histogram_quantile(0.90, sum by(le) (rate(ccip_message_latency_milliseconds_bucket{source_chain="ethereum"}[24h]))) + p99: histogram_quantile(0.99, sum by(le) (rate(ccip_message_latency_milliseconds_bucket{source_chain="ethereum"}[24h]))) + mean: sum(rate(ccip_message_latency_milliseconds_sum{source_chain="ethereum"}[24h])) / sum(rate(ccip_message_latency_milliseconds_count{source_chain="ethereum"}[24h])) + sample_size: sum(increase(ccip_message_seen_total{source_chain="ethereum"}[24h])) + series: histogram_quantile(0.50, sum by(le) (rate(ccip_message_latency_milliseconds_bucket{source_chain="ethereum"}[1h]))) + + - slug: solana + name: Solana + tag: Solana → any chain, DON commits after Solana finalized commitment + formula: "50th percentile over 24h of Chainlink CCIP end-to-end delivery latency for messages sourced from Solana." + queries: + p50: histogram_quantile(0.50, sum by(le) (rate(ccip_message_latency_milliseconds_bucket{source_chain="solana"}[24h]))) + p90: histogram_quantile(0.90, sum by(le) (rate(ccip_message_latency_milliseconds_bucket{source_chain="solana"}[24h]))) + p99: histogram_quantile(0.99, sum by(le) (rate(ccip_message_latency_milliseconds_bucket{source_chain="solana"}[24h]))) + mean: sum(rate(ccip_message_latency_milliseconds_sum{source_chain="solana"}[24h])) / sum(rate(ccip_message_latency_milliseconds_count{source_chain="solana"}[24h])) + sample_size: sum(increase(ccip_message_seen_total{source_chain="solana"}[24h])) + series: histogram_quantile(0.50, sum by(le) (rate(ccip_message_latency_milliseconds_bucket{source_chain="solana"}[1h]))) + + - slug: bnb + name: BNB Chain + tag: BNB Chain → any chain, DON commits after BEP-126 fast-finality + formula: "50th percentile over 24h of Chainlink CCIP end-to-end delivery latency for messages sourced from BNB Chain." + queries: + p50: histogram_quantile(0.50, sum by(le) (rate(ccip_message_latency_milliseconds_bucket{source_chain="bnb"}[24h]))) + p90: histogram_quantile(0.90, sum by(le) (rate(ccip_message_latency_milliseconds_bucket{source_chain="bnb"}[24h]))) + p99: histogram_quantile(0.99, sum by(le) (rate(ccip_message_latency_milliseconds_bucket{source_chain="bnb"}[24h]))) + mean: sum(rate(ccip_message_latency_milliseconds_sum{source_chain="bnb"}[24h])) / sum(rate(ccip_message_latency_milliseconds_count{source_chain="bnb"}[24h])) + sample_size: sum(increase(ccip_message_seen_total{source_chain="bnb"}[24h])) + series: histogram_quantile(0.50, sum by(le) (rate(ccip_message_latency_milliseconds_bucket{source_chain="bnb"}[1h]))) + + - slug: polygon + name: Polygon + tag: Polygon PoS → any chain, DON commits after Bor block confirmation + formula: "50th percentile over 24h of Chainlink CCIP end-to-end delivery latency for messages sourced from Polygon." + queries: + p50: histogram_quantile(0.50, sum by(le) (rate(ccip_message_latency_milliseconds_bucket{source_chain="polygon"}[24h]))) + p90: histogram_quantile(0.90, sum by(le) (rate(ccip_message_latency_milliseconds_bucket{source_chain="polygon"}[24h]))) + p99: histogram_quantile(0.99, sum by(le) (rate(ccip_message_latency_milliseconds_bucket{source_chain="polygon"}[24h]))) + mean: sum(rate(ccip_message_latency_milliseconds_sum{source_chain="polygon"}[24h])) / sum(rate(ccip_message_latency_milliseconds_count{source_chain="polygon"}[24h])) + sample_size: sum(increase(ccip_message_seen_total{source_chain="polygon"}[24h])) + series: histogram_quantile(0.50, sum by(le) (rate(ccip_message_latency_milliseconds_bucket{source_chain="polygon"}[1h]))) + + - slug: avalanche + name: Avalanche + tag: Avalanche C-Chain → any chain, DON commits after Snowman++ finality + formula: "50th percentile over 24h of Chainlink CCIP end-to-end delivery latency for messages sourced from Avalanche." + queries: + p50: histogram_quantile(0.50, sum by(le) (rate(ccip_message_latency_milliseconds_bucket{source_chain="avalanche"}[24h]))) + p90: histogram_quantile(0.90, sum by(le) (rate(ccip_message_latency_milliseconds_bucket{source_chain="avalanche"}[24h]))) + p99: histogram_quantile(0.99, sum by(le) (rate(ccip_message_latency_milliseconds_bucket{source_chain="avalanche"}[24h]))) + mean: sum(rate(ccip_message_latency_milliseconds_sum{source_chain="avalanche"}[24h])) / sum(rate(ccip_message_latency_milliseconds_count{source_chain="avalanche"}[24h])) + sample_size: sum(increase(ccip_message_seen_total{source_chain="avalanche"}[24h])) + series: histogram_quantile(0.50, sum by(le) (rate(ccip_message_latency_milliseconds_bucket{source_chain="avalanche"}[1h]))) + + - slug: arbitrum + name: Arbitrum + tag: Arbitrum One → any chain, DON commits after inherited Ethereum finality + formula: "50th percentile over 24h of Chainlink CCIP end-to-end delivery latency for messages sourced from Arbitrum." + queries: + p50: histogram_quantile(0.50, sum by(le) (rate(ccip_message_latency_milliseconds_bucket{source_chain="arbitrum"}[24h]))) + p90: histogram_quantile(0.90, sum by(le) (rate(ccip_message_latency_milliseconds_bucket{source_chain="arbitrum"}[24h]))) + p99: histogram_quantile(0.99, sum by(le) (rate(ccip_message_latency_milliseconds_bucket{source_chain="arbitrum"}[24h]))) + mean: sum(rate(ccip_message_latency_milliseconds_sum{source_chain="arbitrum"}[24h])) / sum(rate(ccip_message_latency_milliseconds_count{source_chain="arbitrum"}[24h])) + sample_size: sum(increase(ccip_message_seen_total{source_chain="arbitrum"}[24h])) + series: histogram_quantile(0.50, sum by(le) (rate(ccip_message_latency_milliseconds_bucket{source_chain="arbitrum"}[1h]))) + + - slug: base + name: Base + tag: Base → any chain, DON commits after L2 block + inherited Ethereum finality + formula: "50th percentile over 24h of Chainlink CCIP end-to-end delivery latency for messages sourced from Base." + queries: + p50: histogram_quantile(0.50, sum by(le) (rate(ccip_message_latency_milliseconds_bucket{source_chain="base"}[24h]))) + p90: histogram_quantile(0.90, sum by(le) (rate(ccip_message_latency_milliseconds_bucket{source_chain="base"}[24h]))) + p99: histogram_quantile(0.99, sum by(le) (rate(ccip_message_latency_milliseconds_bucket{source_chain="base"}[24h]))) + mean: sum(rate(ccip_message_latency_milliseconds_sum{source_chain="base"}[24h])) / sum(rate(ccip_message_latency_milliseconds_count{source_chain="base"}[24h])) + sample_size: sum(increase(ccip_message_seen_total{source_chain="base"}[24h])) + series: histogram_quantile(0.50, sum by(le) (rate(ccip_message_latency_milliseconds_bucket{source_chain="base"}[1h]))) + + - slug: optimism + name: Optimism + tag: Optimism → any chain, DON commits after L2 block + inherited Ethereum finality + formula: "50th percentile over 24h of Chainlink CCIP end-to-end delivery latency for messages sourced from Optimism." + queries: + p50: histogram_quantile(0.50, sum by(le) (rate(ccip_message_latency_milliseconds_bucket{source_chain="optimism"}[24h]))) + p90: histogram_quantile(0.90, sum by(le) (rate(ccip_message_latency_milliseconds_bucket{source_chain="optimism"}[24h]))) + p99: histogram_quantile(0.99, sum by(le) (rate(ccip_message_latency_milliseconds_bucket{source_chain="optimism"}[24h]))) + mean: sum(rate(ccip_message_latency_milliseconds_sum{source_chain="optimism"}[24h])) / sum(rate(ccip_message_latency_milliseconds_count{source_chain="optimism"}[24h])) + sample_size: sum(increase(ccip_message_seen_total{source_chain="optimism"}[24h])) + series: histogram_quantile(0.50, sum by(le) (rate(ccip_message_latency_milliseconds_bucket{source_chain="optimism"}[1h]))) diff --git a/harnesses/chainlink-ccip-latency/Dockerfile b/harnesses/chainlink-ccip-latency/Dockerfile new file mode 100644 index 00000000..270b0b69 --- /dev/null +++ b/harnesses/chainlink-ccip-latency/Dockerfile @@ -0,0 +1,22 @@ +FROM golang:1.24-alpine AS builder + +WORKDIR /app +RUN apk add --no-cache git + +COPY go.mod ./ +RUN go mod download || true + +COPY . . + +RUN CGO_ENABLED=0 GOOS=linux go build -o /app/monitor ./cmd/script + +FROM debian:bookworm-slim + +WORKDIR /app +RUN apt-get update && apt-get install -y ca-certificates && rm -rf /var/lib/apt/lists/* + +COPY --from=builder /app/monitor /app/monitor + +EXPOSE 2112 + +CMD ["/app/monitor"] diff --git a/harnesses/chainlink-ccip-latency/cmd/script/chains.go b/harnesses/chainlink-ccip-latency/cmd/script/chains.go new file mode 100644 index 00000000..e551d3c4 --- /dev/null +++ b/harnesses/chainlink-ccip-latency/cmd/script/chains.go @@ -0,0 +1,63 @@ +package main + +// ccipChainSlug normalises CCIP's `sourceNetworkInfo.name` to the OCB +// canonical chain slug used across every other bench. +// +// CCIP uses a hierarchical naming convention that reflects the L1/L2 +// relationship: L1s are `-mainnet` (ethereum-mainnet, +// polygon-mainnet, avalanche-mainnet) while L2s are +// `-mainnet--1` (ethereum-mainnet-base-1, +// ethereum-mainnet-arbitrum-1, bitcoin-mainnet-bitlayer-1). We +// enumerate both forms explicitly rather than parse the pattern — +// makes the mapping legible and the audit trail obvious when CCIP +// adds a new chain we didn't anticipate. +// +// Only mainnet entries are mapped; testnet rows are dropped in main.go +// via the `environment != "mainnet"` guard so we never emit test-chain +// latency. +// +// Fall-through in main.go emits `chain-` for unknown names so +// the histogram flows under a synthetic label until we add the row here. +var ccipChainSlug = map[string]string{ + // L1s + "ethereum-mainnet": "ethereum", + "solana-mainnet": "solana", + "binance_smart_chain-mainnet": "bnb", + "polygon-mainnet": "polygon", + "avalanche-mainnet": "avalanche", + "celo-mainnet": "celo", + "gnosis-mainnet": "gnosis", + "fantom-mainnet": "fantom", + "metis-mainnet": "metis", + "ronin-mainnet": "ronin", + "soneium-mainnet": "soneium", + "monad-mainnet": "monad", + "berachain-mainnet": "berachain", + "apechain-mainnet": "apechain", + "corn-mainnet": "corn", + "sonic-mainnet": "sonic", + "robinhood-mainnet": "robinhood", + "0g-mainnet": "chain-0g", + "adi-mainnet": "chain-adi", + + // Ethereum L2s (hierarchical naming) + "ethereum-mainnet-arbitrum-1": "arbitrum", + "ethereum-mainnet-base-1": "base", + "ethereum-mainnet-optimism-1": "optimism", + "ethereum-mainnet-ink-1": "ink", + "ethereum-mainnet-mantle-1": "mantle", + "ethereum-mainnet-scroll-1": "scroll", + "ethereum-mainnet-blast-1": "blast", + "ethereum-mainnet-linea-1": "linea", + "ethereum-mainnet-zksync-1": "zksync", + "ethereum-mainnet-world-1": "world-chain", + "ethereum-mainnet-unichain-1": "unichain", + "ethereum-mainnet-bob-1": "bob", + "ethereum-mainnet-kroma-1": "kroma", + + // Bitcoin L2s + "bitcoin-mainnet-bitlayer-1": "bitlayer", + + // Polkadot parachains + "polkadot-mainnet-astar": "astar", +} diff --git a/harnesses/chainlink-ccip-latency/cmd/script/main.go b/harnesses/chainlink-ccip-latency/cmd/script/main.go new file mode 100644 index 00000000..19b6e3c8 --- /dev/null +++ b/harnesses/chainlink-ccip-latency/cmd/script/main.go @@ -0,0 +1,253 @@ +// chainlink-ccip-latency: passive Chainlink CCIP message latency bench. +// +// Every 60s, polls https://api.ccip.chain.link/v2/messages?limit=100 for +// the most recent CCIP messages. For each message the monitor has not +// seen before AND that has completed (`status == SUCCESS`), computes +// `receiptTimestamp - sendTimestamp` and records it in a Prometheus +// histogram bucketed by source + destination chain. +// +// The harness never sends canary transactions — CCIP handles enough +// production traffic (several hundred messages per day across the top +// corridors) that passive polling gives statistically robust p50/p90 +// per corridor inside a 24-hour window. +// +// Design notes: +// - We filter out `environment != "mainnet"` before observing. +// Testnet lanes have wildly different profiles and would poison the +// leaderboard. +// - Statuses other than SUCCESS (SENT, SOURCE_FINALIZED, COMMITTED, +// FAILED, ...) are counted separately as `ccip_message_status_total` +// so a spike in COMMITTED without matching SUCCESS surfaces a +// stuck-execution incident without polluting latency data. +// - CCIP intentionally waits for source-chain finality before its DON +// commits. Ethereum-source lanes therefore include ~13min of +// finality wait. We expose this in the raw histogram; the fair +// comparison to Wormhole (finality-agnostic) lives in the meta-bench. +// - Dedupe on `messageId` via a 10k-entry LRU. Sustained rate is +// ~25 msg/min across the network, so the cache holds ~7h of history. +package main + +import ( + "container/list" + "context" + "encoding/json" + "fmt" + "io" + "log" + "net/http" + "os" + "os/signal" + "strings" + "sync" + "syscall" + "time" + + "github.com/prometheus/client_golang/prometheus/promhttp" +) + +const ( + ccipMessagesURL = "https://api.ccip.chain.link/v2/messages?limit=100" + pollInterval = 60 * time.Second + requestTimeout = 15 * time.Second + dedupeCacheMaxLen = 10_000 + metricsListenAddr = ":2112" + // Guard against clock skew or backfilled rows: any latency > 2h is + // almost certainly noise (a stuck message re-executed later, or a + // timestamp typo). Kept generous because CCIP finality wait on ETH + // legitimately reaches ~25 min in bad epochs. + maxLatencyMs = 7_200_000 // 2h +) + +type ccipNetwork struct { + Name string `json:"name"` + DisplayName string `json:"displayName"` + Environment string `json:"environment"` +} + +type ccipMessage struct { + MessageID string `json:"messageId"` + Status string `json:"status"` + SourceNetworkInfo ccipNetwork `json:"sourceNetworkInfo"` + DestNetworkInfo ccipNetwork `json:"destNetworkInfo"` + SendTimestamp string `json:"sendTimestamp"` + ReceiptTimestamp *string `json:"receiptTimestamp"` +} + +type ccipResponse struct { + Data []ccipMessage `json:"data"` +} + +// lruSet keeps the last N inserted keys, evicting FIFO. Not thread-safe — +// caller (single-goroutine poller) synchronises. +type lruSet struct { + max int + order *list.List + set map[string]*list.Element +} + +func newLRUSet(max int) *lruSet { + return &lruSet{max: max, order: list.New(), set: make(map[string]*list.Element, max)} +} + +func (l *lruSet) contains(k string) bool { _, ok := l.set[k]; return ok } + +func (l *lruSet) add(k string) { + if _, ok := l.set[k]; ok { + return + } + el := l.order.PushBack(k) + l.set[k] = el + for l.order.Len() > l.max { + front := l.order.Front() + if front == nil { + return + } + l.order.Remove(front) + delete(l.set, front.Value.(string)) + } +} + +func (l *lruSet) size() int { return l.order.Len() } + +// chainSlugFromCCIPName maps CCIP's kebab-case network name to the OCB +// canonical chain slug. Unknown names get a `chain-` fallback +// so we never drop data silently — the label just shows up as a new row +// and we add the mapping to chains.go next release. +func chainSlugFromCCIPName(name string) string { + if s, ok := ccipChainSlug[name]; ok { + return s + } + // Strip "-mainnet" suffix, lowercase, dashes preserved. + name = strings.TrimSuffix(name, "-mainnet") + name = strings.ToLower(name) + return "chain-" + name +} + +func poll(ctx context.Context, client *http.Client, seen *lruSet) error { + req, err := http.NewRequestWithContext(ctx, "GET", ccipMessagesURL, nil) + if err != nil { + return fmt.Errorf("request: %w", err) + } + req.Header.Set("Accept", "application/json") + req.Header.Set("User-Agent", "OpenChainBench/chainlink-ccip-latency") + resp, err := client.Do(req) + if err != nil { + return fmt.Errorf("http: %w", err) + } + defer resp.Body.Close() + if resp.StatusCode != 200 { + body, _ := io.ReadAll(io.LimitReader(resp.Body, 200)) + return fmt.Errorf("status %d: %s", resp.StatusCode, string(body)) + } + var parsed ccipResponse + if err := json.NewDecoder(resp.Body).Decode(&parsed); err != nil { + return fmt.Errorf("decode: %w", err) + } + fresh := 0 + success := 0 + for _, m := range parsed.Data { + if m.MessageID == "" { + continue + } + // Testnet lanes are noise for a production latency bench — + // drop before doing anything else so they don't touch the + // status counter either. + if m.SourceNetworkInfo.Environment != "mainnet" || + m.DestNetworkInfo.Environment != "mainnet" { + continue + } + + srcSlug := chainSlugFromCCIPName(m.SourceNetworkInfo.Name) + dstSlug := chainSlugFromCCIPName(m.DestNetworkInfo.Name) + + // Count every status we see (once per messageId), even before + // the message reaches SUCCESS, so the stuck-pipeline signal + // surfaces in Prom without waiting for completion. + if !seen.contains(m.MessageID) { + ccipStatusTotal.WithLabelValues(srcSlug, m.Status).Inc() + } + + if m.Status != "SUCCESS" || m.ReceiptTimestamp == nil { + // Do NOT add to dedupe yet — we want to observe the + // SUCCESS transition when it eventually appears. + continue + } + if seen.contains(m.MessageID) { + continue + } + + sendTs, err1 := time.Parse(time.RFC3339, m.SendTimestamp) + recvTs, err2 := time.Parse(time.RFC3339, *m.ReceiptTimestamp) + if err1 != nil || err2 != nil { + // Malformed timestamps — mark seen so we don't retry + // forever, but skip the histogram observation. + seen.add(m.MessageID) + continue + } + deltaMs := float64(recvTs.Sub(sendTs).Milliseconds()) + if deltaMs < 0 || deltaMs > maxLatencyMs { + seen.add(m.MessageID) + continue + } + ccipLatencyMs.WithLabelValues(srcSlug, dstSlug).Observe(deltaMs) + ccipSeenTotal.WithLabelValues(srcSlug, dstSlug).Inc() + seen.add(m.MessageID) + fresh++ + success++ + } + ccipDedupeCacheSize.Set(float64(seen.size())) + log.Printf("poll: %d rows, %d success-fresh, dedupe=%d", len(parsed.Data), success, seen.size()) + return nil +} + +func main() { + log.SetFlags(log.LstdFlags | log.Lmicroseconds) + log.Println("chainlink-ccip-latency: starting") + + ctx, cancel := context.WithCancel(context.Background()) + defer cancel() + + client := &http.Client{Timeout: requestTimeout} + seen := newLRUSet(dedupeCacheMaxLen) + var mu sync.Mutex + + go func() { + http.Handle("/metrics", promhttp.Handler()) + http.HandleFunc("/health", func(w http.ResponseWriter, _ *http.Request) { + w.WriteHeader(200) + _, _ = w.Write([]byte("ok")) + }) + log.Printf("metrics: listening on %s", metricsListenAddr) + if err := http.ListenAndServe(metricsListenAddr, nil); err != nil { + log.Fatalf("metrics server: %v", err) + } + }() + + mu.Lock() + if err := poll(ctx, client, seen); err != nil { + log.Printf("poll error: %v", err) + ccipPollErrors.Inc() + } + mu.Unlock() + + ticker := time.NewTicker(pollInterval) + defer ticker.Stop() + + sig := make(chan os.Signal, 1) + signal.Notify(sig, syscall.SIGINT, syscall.SIGTERM) + + for { + select { + case <-ticker.C: + mu.Lock() + if err := poll(ctx, client, seen); err != nil { + log.Printf("poll error: %v", err) + ccipPollErrors.Inc() + } + mu.Unlock() + case <-sig: + log.Println("shutdown") + return + } + } +} diff --git a/harnesses/chainlink-ccip-latency/cmd/script/metrics.go b/harnesses/chainlink-ccip-latency/cmd/script/metrics.go new file mode 100644 index 00000000..767013bf --- /dev/null +++ b/harnesses/chainlink-ccip-latency/cmd/script/metrics.go @@ -0,0 +1,74 @@ +package main + +import ( + "github.com/prometheus/client_golang/prometheus" +) + +// ccip_message_latency_milliseconds is a histogram of Chainlink CCIP +// end-to-end message latency, bucketed by source chain. +// Latency = `receiptTimestamp - sendTimestamp` from the CCIP Tools API +// `/v2/messages` payload, i.e. source-tx submission → destination +// execution. Emitted in MILLISECONDS to match the site's `unit: ms` +// display convention. +// +// Buckets extend beyond the Wormhole ceiling (30 min) to 90 min +// because CCIP structurally waits for source-chain finality before +// its DON commits: Ethereum-source lanes routinely take 13-15 min +// just for finality and ~21 min end-to-end. Without the extended +// tail, ETH p90/p99 would pin to the 30-min bucket edge (same +// failure mode we hit with Wormhole VAA on 2026-07-24). +// +// A companion label `dest_chain` is emitted so a matrix cell can +// slice by corridor, matching how the meta-bench aggregates it later. +var ( + ccipLatencyMs = prometheus.NewHistogramVec( + prometheus.HistogramOpts{ + Name: "ccip_message_latency_milliseconds", + Help: "End-to-end delivery latency (ms) for a Chainlink CCIP message, from source tx to destination execution, labeled by source + destination chain.", + Buckets: []float64{5_000, 15_000, 30_000, 60_000, 120_000, 180_000, 300_000, 600_000, 900_000, 1_200_000, 1_800_000, 2_400_000, 3_000_000, 3_600_000, 4_500_000, 5_400_000}, + }, + []string{"source_chain", "dest_chain"}, + ) + + ccipSeenTotal = prometheus.NewCounterVec( + prometheus.CounterOpts{ + Name: "ccip_message_seen_total", + Help: "Total unique successful CCIP messages observed by this monitor, by source + destination chain.", + }, + []string{"source_chain", "dest_chain"}, + ) + + // Non-success statuses observed. CCIP messages flow through + // SENT -> SOURCE_FINALIZED -> COMMITTED -> SUCCESS; we only + // histogram SUCCESS but count the others so a spike in COMMITTED + // without SUCCESS surfaces stuck delivery pipelines. + ccipStatusTotal = prometheus.NewCounterVec( + prometheus.CounterOpts{ + Name: "ccip_message_status_total", + Help: "CCIP messages seen per status (SENT, SOURCE_FINALIZED, COMMITTED, SUCCESS, FAILED, ...), by source chain.", + }, + []string{"source_chain", "status"}, + ) + + ccipPollErrors = prometheus.NewCounter( + prometheus.CounterOpts{ + Name: "ccip_message_poll_errors_total", + Help: "Total failed pollings of the CCIP Tools /v2/messages feed.", + }, + ) + + ccipDedupeCacheSize = prometheus.NewGauge( + prometheus.GaugeOpts{ + Name: "ccip_message_dedupe_cache_size", + Help: "Number of recent CCIP message ids the monitor keeps for deduplication.", + }, + ) +) + +func init() { + prometheus.MustRegister(ccipLatencyMs) + prometheus.MustRegister(ccipSeenTotal) + prometheus.MustRegister(ccipStatusTotal) + prometheus.MustRegister(ccipPollErrors) + prometheus.MustRegister(ccipDedupeCacheSize) +} diff --git a/harnesses/chainlink-ccip-latency/go.mod b/harnesses/chainlink-ccip-latency/go.mod new file mode 100644 index 00000000..62f5d02d --- /dev/null +++ b/harnesses/chainlink-ccip-latency/go.mod @@ -0,0 +1,18 @@ +module chainlink-ccip-latency + +go 1.24.0 + +require github.com/prometheus/client_golang v1.23.2 + +require ( + github.com/beorn7/perks v1.0.1 // indirect + github.com/cespare/xxhash/v2 v2.3.0 // indirect + github.com/kr/text v0.2.0 // indirect + github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect + github.com/prometheus/client_model v0.6.2 // indirect + github.com/prometheus/common v0.66.1 // indirect + github.com/prometheus/procfs v0.17.0 // indirect + go.yaml.in/yaml/v2 v2.4.2 // indirect + golang.org/x/sys v0.36.0 // indirect + google.golang.org/protobuf v1.36.10 // indirect +) diff --git a/harnesses/chainlink-ccip-latency/go.sum b/harnesses/chainlink-ccip-latency/go.sum new file mode 100644 index 00000000..f415d46b --- /dev/null +++ b/harnesses/chainlink-ccip-latency/go.sum @@ -0,0 +1,46 @@ +github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM= +github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw= +github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs= +github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= +github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E= +github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= +github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8= +github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU= +github.com/klauspost/compress v1.18.0 h1:c/Cqfb0r+Yi+JtIEq73FWXVkRonBlf0CRNYc8Zttxdo= +github.com/klauspost/compress v1.18.0/go.mod h1:2Pp+KzxcywXVXMr50+X0Q/Lsb43OQHYWRCY2AiWywWQ= +github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE= +github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk= +github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= +github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE= +github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc= +github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw= +github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA= +github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ= +github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= +github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= +github.com/prometheus/client_golang v1.23.2 h1:Je96obch5RDVy3FDMndoUsjAhG5Edi49h0RJWRi/o0o= +github.com/prometheus/client_golang v1.23.2/go.mod h1:Tb1a6LWHB3/SPIzCoaDXI4I8UHKeFTEQ1YCr+0Gyqmg= +github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNwqPLxwZyk= +github.com/prometheus/client_model v0.6.2/go.mod h1:y3m2F6Gdpfy6Ut/GBsUqTWZqCUvMVzSfMLjcu6wAwpE= +github.com/prometheus/common v0.66.1 h1:h5E0h5/Y8niHc5DlaLlWLArTQI7tMrsfQjHV+d9ZoGs= +github.com/prometheus/common v0.66.1/go.mod h1:gcaUsgf3KfRSwHY4dIMXLPV0K/Wg1oZ8+SbZk/HH/dA= +github.com/prometheus/procfs v0.17.0 h1:FuLQ+05u4ZI+SS/w9+BWEM2TXiHKsUQ9TADiRH7DuK0= +github.com/prometheus/procfs v0.17.0/go.mod h1:oPQLaDAMRbA+u8H5Pbfq+dl3VDAvHxMUOVhe0wYB2zw= +github.com/rogpeppe/go-internal v1.10.0 h1:TMyTOH3F/DB16zRVcYyreMH6GnZZrwQVAoYjRBZyWFQ= +github.com/rogpeppe/go-internal v1.10.0/go.mod h1:UQnix2H7Ngw/k4C5ijL5+65zddjncjaFoBhdsK/akog= +github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U= +github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U= +go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto= +go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE= +go.yaml.in/yaml/v2 v2.4.2 h1:DzmwEr2rDGHl7lsFgAHxmNz/1NlQ7xLIrlN2h5d1eGI= +go.yaml.in/yaml/v2 v2.4.2/go.mod h1:081UH+NErpNdqlCXm3TtEran0rJZGxAYx9hb/ELlsPU= +golang.org/x/sys v0.36.0 h1:KVRy2GtZBrk1cBYA7MKu5bEZFxQk4NIDV6RLVcC8o0k= +golang.org/x/sys v0.36.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks= +google.golang.org/protobuf v1.36.10 h1:AYd7cD/uASjIL6Q9LiTjz8JLcrh/88q5UObnmY3aOOE= +google.golang.org/protobuf v1.36.10/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco= +gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk= +gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q= +gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= +gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= diff --git a/src/components/bench-infobox.tsx b/src/components/bench-infobox.tsx index d1f45286..4c09373e 100644 --- a/src/components/bench-infobox.tsx +++ b/src/components/bench-infobox.tsx @@ -42,6 +42,21 @@ export function BenchInfobox({ benchmark }: { benchmark: Benchmark }) { itemScope itemType="https://schema.org/Dataset" > + {/* Required Dataset microdata fields not surfaced in the visible + key/value grid below. Google parses the microdata alongside + the JSON-LD block and validates each Dataset in isolation, so + the microdata Dataset needs name/description/creator or it + flags "Unnamed item, missing name/description" (Rich Results + Test 2026-07-25 on ws-head-latency-* + xstocks-peg). Hidden + lets the microdata graph stay complete without adding + duplicate visible text to the human infobox. */} + + + + + + +
- + + + /api/stat/{benchmark.slug} + +