diff --git a/benchmarks/aggregator-head-lag.yml b/benchmarks/aggregator-head-lag.yml index 39fdfe0c..e72b0c81 100644 --- a/benchmarks/aggregator-head-lag.yml +++ b/benchmarks/aggregator-head-lag.yml @@ -5,7 +5,7 @@ number: "001" title: Fastest crypto price API, live head lag across Mobula, Codex, GeckoTerminal seo_title: "Fastest crypto price API 2026" seo_description: "{{best_name}} leads fastest crypto price API at {{best_p50}} (p50, 24h). Mobula WebSocket, Codex GraphQL, GeckoTerminal REST live across Base, BNB, Solana." -subtitle: Wall-clock head lag in seconds from on-chain swap event to API emission, measured live for Mobula, Codex and GeckoTerminal on Base, BNB Chain and Solana. +subtitle: Wall-clock head lag in seconds from on-chain swap event to API emission, measured live for Mobula, Codex and GeckoTerminal on Base, BNB Chain, Solana and Robinhood Chain. per_chain_explainer: - slug: base @@ -20,6 +20,10 @@ per_chain_explainer: h2: "Lowest Solana price feed head lag" body: | {{best_name:chain:solana}} currently leads Solana head lag at {{best_p50:chain:solana}} (p50, 24h). Solana runs a 400 ms slot under Proof of History with confirmed status at typically 1-2 slots, so any second-scale provider latency lives entirely in the indexer pipeline rather than in the chain. Providers that subscribe to Yellowstone gRPC streams of confirmed blocks beat REST-pollers by definition. Measured against a canonical archive node from us-east, eu-west and Singapore. + - slug: robinhood + h2: "Lowest Robinhood Chain price feed head lag" + body: | + {{best_name:chain:robinhood}} currently leads Robinhood Chain head lag at {{best_p50:chain:robinhood}} (p50, 24h). Robinhood Chain is an Arbitrum Orbit L2 with sub-second block cadence, so nearly all of the latency you see here is provider pipeline (indexer, fan-out, edge) rather than chain time. It is also the youngest chain on this bench: indexer coverage is uneven across providers, which makes the leader-to-trailer spread wider than on mature chains. Measured on the highest-activity pool (USDG/WETH, about 40k swaps per day) from us-east, eu-west and Singapore. category: Aggregators status: live @@ -35,7 +39,7 @@ seo_intro: | number. Mobula's WebSocket, Codex's GraphQL feed and GeckoTerminal's REST endpoint are watched from three regions (us-east, eu-west, sgp) against canonical-tip archive - nodes on Base, BNB Chain and Solana. Because each chain has its own + nodes on Base, BNB Chain, Solana and Robinhood Chain. Because each chain has its own base block cadence (Solana's ~400 ms slots vs Base/BNB's 2-3 s blocks), the headline winner shifts by chain. On Solana the leader is {{best_name:chain:solana}} at {{best_p50:chain:solana}}; on Base @@ -51,19 +55,19 @@ seo_intro: | abstract: | We measure the gap between a transaction settling on chain and the same transaction appearing on each aggregator's data feed. The harness watches - reference pools across Base, BNB Chain and Solana from three regions + reference pools across Base, BNB Chain, Solana and Robinhood Chain from three regions (us-east, eu-west, sgp), records every new on-chain event with millisecond timestamps, and observes when each aggregator's WebSocket reflects the same event. methodology: - "Aggregators measured: Mobula, Codex, GeckoTerminal." - - "Chains: Base, BNB Chain, Solana." + - "Chains: Base, BNB Chain, Solana, Robinhood Chain." - "Regions: us-east, eu-west, sgp. Cross-region median reported in the headline." - "Reference: archive nodes per chain, validated against block hashes." - "Metric: gauge `head_lag_seconds`, sampled every 15 seconds. Aggregated over the 24-hour window using `quantile_over_time`." - "Success rate: presence ratio. Share of expected sampling slots where a value was actually emitted (5,760 expected per provider per day at the 15s cadence). 100% means the aggregator's feed was reachable for the full window." - - "Cardinality: 3 aggregators × 3 chains × 3 regions = 27 active series." + - "Cardinality: 3 aggregators × 4 chains × 3 regions = 36 active series." findings: - "Across the unfiltered cross-chain aggregate the leader is {{best_name}} at {{best_p50}} (p50, 24 h) over {{count}} measured providers, but that headline number mixes chains with very different baseline cadences (Solana 400 ms slots vs Base/BNB 2-3 s blocks) so the per-chain breakdown below is what to read for an integration decision." @@ -90,7 +94,7 @@ faq: - q: "How fast does GeckoTerminal update prices?" a: "{{name:geckoterminal}} currently sits at {{p50:geckoterminal}} (p50, 24 h) head lag on this benchmark. REST endpoints update at the publisher's chosen cadence rather than the moment a swap lands, which is the architectural reason REST polling reads slower than WebSocket feeds for the same underlying chain data." - q: "How is head lag measured on OpenChainBench?" - a: "The harness holds a live WebSocket subscription to canonical-tip archive nodes on each chain (Base, BNB Chain, Solana). Every new on-chain swap is timestamped on receipt with millisecond precision. The harness then watches each provider's feed for the same event and computes the delta. p50, p90 and p99 are aggregated over 24 hours via Prometheus `quantile_over_time`. Errors and dropouts are tracked separately as a presence ratio so a fast feed that disappears does not silently rank above a slower feed that stays up." + a: "The harness holds a live WebSocket subscription to canonical-tip archive nodes on each chain (Base, BNB Chain, Solana, Robinhood Chain). Every new on-chain swap is timestamped on receipt with millisecond precision. The harness then watches each provider's feed for the same event and computes the delta. p50, p90 and p99 are aggregated over 24 hours via Prometheus `quantile_over_time`. Errors and dropouts are tracked separately as a presence ratio so a fast feed that disappears does not silently rank above a slower feed that stays up." - q: "Which data provider should I pick for a real-time trading interface?" a: "Read both the p50 and the p99 columns. p50 tells the typical experience, p99 tells the worst minute a user will see when a chain spikes or a region's path degrades. A 500 ms p50 with a 30 s p99 means most of the time the UI feels snappy and once per minute it stutters. The leaderboard splits both, plus the success rate column flags providers whose feed dropped during the measurement window." @@ -99,10 +103,11 @@ faq: # every PromQL selector via lib/spec.ts:injectChainFilter. dimensions: chain: - - { value: all, label: All chains } - - { value: base, label: Base } - - { value: bnb, label: BNB Chain } - - { value: solana, label: Solana } + - { value: all, label: All chains } + - { value: base, label: Base } + - { value: bnb, label: BNB Chain } + - { value: solana, label: Solana } + - { value: robinhood, label: Robinhood } # Region values match the raw harness labels (REGION env on each # Railway probe: us-east / eu-west / sgp). Declared so the # /compare/[slug] page can render a per chain x per region matrix diff --git a/harnesses/aggregator-head-lag/cmd/script/geckoterminal_monitor.go b/harnesses/aggregator-head-lag/cmd/script/geckoterminal_monitor.go index f47856f0..89d28b70 100644 --- a/harnesses/aggregator-head-lag/cmd/script/geckoterminal_monitor.go +++ b/harnesses/aggregator-head-lag/cmd/script/geckoterminal_monitor.go @@ -52,6 +52,15 @@ var geckoTerminalPools = []struct { PoolID: "160787671", Chain: "bnb", }, + { + // Highest txn-frequency pool on Robinhood Chain (~40k swaps/24h). + // Internal GT pool_id source: app.geckoterminal.com/api/p1/robinhood/pools/
+ // Pool address: 0x69bfaf19c9f377bb306a89aed9f6b07e2c1a8d9a + Name: "USDG/WETH Robinhood", + Network: "robinhood", + PoolID: "328167634", + Chain: "robinhood", + }, } // ActionCable message structures diff --git a/harnesses/aggregator-head-lag/cmd/script/head_lag_monitor.go b/harnesses/aggregator-head-lag/cmd/script/head_lag_monitor.go index 3a8494fa..b52a9f12 100644 --- a/harnesses/aggregator-head-lag/cmd/script/head_lag_monitor.go +++ b/harnesses/aggregator-head-lag/cmd/script/head_lag_monitor.go @@ -16,6 +16,14 @@ import ( // Measures indexation latency: time between on-chain event and WebSocket receipt // ============================================================================ +// Post-reconnect grace period: Mobula WS replays buffered/old trades right +// after a (re)connect, which show up as multi-second "lag" that is NOT real +// indexation latency. During this window we do NOT feed the alerting gauges +// (mobula_head_lag_detailed_seconds & co) so #alerting-aggregator-latency +// stays quiet on reconnect bursts. The head_lag_seconds bench gauge keeps +// being fed (no spike alert on it; keeps AggregatorHeadLagStale happy). +const reconnectGracePeriod = 10 * time.Second + // Pool configurations for head lag monitoring type HeadLagPool struct { Name string // Human readable name @@ -28,16 +36,10 @@ type HeadLagPool struct { // Pools to monitor - high activity pools for accurate lag measurement var headLagPools = []HeadLagPool{ { - // Switched 2026-05-29 to the Raydium SOL/USDC ($8.7M liq, $2.6M - // vol24h vs the previous Orca pool 7qbRF6Y… at $206K / $46K). - // This is the only Solana pool GMGN pushes per-pair events for, - // so all 4 providers measure the same reference pool → true - // apples-to-apples instead of GMGN-on-slot-heartbeat vs others- - // on-per-swap. Bonus: much higher event rate tightens stats. Name: "SOL/USDC Raydium", Blockchain: "solana", NetworkID: 1399811149, - Address: "58oQChx4yWmvKdwLLZzBi4ChoCc2fqCUWBkwMihLYQo2", + Address: "7qbRF6YsyGuLUVs6Y1q64bdVrfe4ZcUUz1JRdoVNUJnm", ChainName: "solana", }, { @@ -48,19 +50,20 @@ var headLagPools = []HeadLagPool{ ChainName: "base", }, { - // Switched 2026-05-29 off the WBNB/BUSD pool ($104K vol24h — - // BUSD was deprecated by Binance in 2023 so swap events on it - // are too sparse for percentile stats: the bench was reporting - // zero BNB samples across all 3 regions). New pool is - // PancakeSwap V3 WBNB/USDT 0.01% with $17M liq + $100M vol24h — - // ~1000x event rate, the canonical BNB pair every aggregator - // indexes. - Name: "WBNB/USDT PancakeSwap V3", + Name: "WBNB/BUSD PancakeSwap", Blockchain: "evm:56", NetworkID: 56, - Address: "0x172fcd41e0913e95784454622d1c3724f546f849", + Address: "0x58f876857a02d6762e0101bb5c46a8c1ed44dc16", ChainName: "bnb", }, + { + // Highest txn-frequency pool on Robinhood Chain (~40k swaps/24h). + Name: "USDG/WETH Robinhood", + Blockchain: "evm:4663", + NetworkID: 4663, + Address: "0x69bfaf19c9f377bb306a89aed9f6b07e2c1a8d9a", + ChainName: "robinhood", + }, } // ============================================================================ @@ -97,9 +100,11 @@ func runMobulaHeadLagMonitor(config *Config, stopChan <-chan struct{}, wg *sync. return default: err := connectAndMonitorMobula(config, stopChan) + RecordWSConnected("mobula", config.MonitorRegion, false) if err != nil { - log.Printf("[HEAD-LAG][MOBULA] Connection error: %v. Reconnecting in %v...", err, reconnectDelay) - + RecordWSReconnect("mobula", config.MonitorRegion) + log.Printf("[HEAD-LAG][MOBULA] 🔌 DISCONNECTED: %v. Reconnecting in %v...", err, reconnectDelay) + select { case <-stopChan: return @@ -149,7 +154,9 @@ func connectAndMonitorMobula(config *Config, stopChan <-chan struct{}) error { return fmt.Errorf("subscribe failed: %w", err) } - fmt.Printf("[HEAD-LAG][MOBULA] Subscribed to %d pools\n", len(items)) + connectedAt := time.Now() + RecordWSConnected("mobula", config.MonitorRegion, true) + log.Printf("[HEAD-LAG][MOBULA] ✅ CONNECTED — subscribed to %d pools (alert-gauge grace period: %v)", len(items), reconnectGracePeriod) // Start ping goroutine pingDone := make(chan struct{}) @@ -223,19 +230,28 @@ func connectAndMonitorMobula(config *Config, stopChan <-chan struct{}) error { // Track latest tx per pool so alert annotations can link to it RecordMobulaLastTx(chainName, config.MonitorRegion, trade.Pair, trade.Hash) - // Record detailed metric with breakdown - RecordMobulaHeadLagDetailed( - chainName, - config.MonitorRegion, - trade.Pair, - trade.Hash, - totalLagMs, - mobulaLagMs, - networkLagMs, - onChainTime.Format("2006-01-02T15:04:05Z"), - mobulaProcessTime.Format("2006-01-02T15:04:05Z"), - receiveTime.Format("2006-01-02T15:04:05Z"), - ) + // Post-reconnect grace: don't feed the alerting gauges with + // replayed/buffered trades from the fresh connection. + if time.Since(connectedAt) < reconnectGracePeriod { + if totalLagMs > 2000 { + log.Printf("[HEAD-LAG][MOBULA][%s] ⏭️ GRACE: late trade %.2fs within %v of reconnect — alert gauge NOT updated | Tx: %s", + chainName, lagSeconds, reconnectGracePeriod, trade.Hash) + } + } else { + // Record detailed metric with breakdown + RecordMobulaHeadLagDetailed( + chainName, + config.MonitorRegion, + trade.Pair, + trade.Hash, + totalLagMs, + mobulaLagMs, + networkLagMs, + onChainTime.Format("2006-01-02T15:04:05Z"), + mobulaProcessTime.Format("2006-01-02T15:04:05Z"), + receiveTime.Format("2006-01-02T15:04:05Z"), + ) + } // Enhanced logging for spikes if totalLagMs > 3000 { @@ -269,6 +285,8 @@ func getChainNameFromBlockchain(blockchain string) string { return "base" case "BNB Smart Chain (BEP20)", "BSC", "evm:56": return "bnb" + case "Robinhood Chain", "evm:4663": + return "robinhood" default: return blockchain } @@ -338,7 +356,9 @@ func runCodexHeadLagMonitor(config *Config, stopChan <-chan struct{}, wg *sync.W log.Printf("[HEAD-LAG][CODEX] 🔄 Connection attempt #%d (proxy rotation enabled)", attemptNum) err := connectAndMonitorCodex(config, stopChan) + RecordWSConnected("codex", config.MonitorRegion, false) if err != nil { + RecordWSReconnect("codex", config.MonitorRegion) consecutiveFailures++ log.Printf("[HEAD-LAG][CODEX] ❌ Connection attempt #%d failed (%d consecutive): %v", attemptNum, consecutiveFailures, err) @@ -446,15 +466,18 @@ func connectAndMonitorCodex(config *Config, stopChan <-chan struct{}) error { var subMsg map[string]interface{} - // Solana: use onUnconfirmedEventsCreated (pre-finalized, lowest latency) + // Solana: commitmentLevel Processed (pre-finalized, lowest latency). + // This replaces onUnconfirmedEventsCreated, deprecated by Codex on + // 2026-04-20 and enforcement-killed for API keys ~2026-07-06 (hard + // WS close 1006). Verified delivering on the JWT path 2026-07-09. if pool.ChainName == "solana" { poolID := fmt.Sprintf("%s:%d", pool.Address, pool.NetworkID) subMsg = map[string]interface{}{ "type": "subscribe", "id": subID, "payload": map[string]interface{}{ - "query": `subscription OnPoolEvents($id: String!) { - onUnconfirmedEventsCreated(id: $id, quoteToken: token0) { + "query": `subscription OnPoolEvents($id: String!, $cl: [EventCommitmentLevel!]) { + onEventsCreated(id: $id, commitmentLevel: $cl) { address networkId events { @@ -467,6 +490,7 @@ func connectAndMonitorCodex(config *Config, stopChan <-chan struct{}) error { }`, "variables": map[string]interface{}{ "id": poolID, + "cl": []string{"Processed"}, }, }, } @@ -505,6 +529,7 @@ func connectAndMonitorCodex(config *Config, stopChan <-chan struct{}) error { log.Printf("[HEAD-LAG][CODEX] ✅ Subscribed to %d pools", len(headLagPools)) log.Printf("[HEAD-LAG][CODEX] 🎉 Connection fully established! Waiting for events...") + RecordWSConnected("codex", config.MonitorRegion, true) // graphql-transport-ws keepalive: send {"type":"ping"} every 20s. // Without this, the rotating proxy (or Codex itself) silently kills idle @@ -512,6 +537,49 @@ func connectAndMonitorCodex(config *Config, stopChan <-chan struct{}) error { // Writes to the websocket conn are serialized via writeMu since we now have // two goroutines writing (this ping loop + the subscribe path on next reconnect). var writeMu sync.Mutex + + // Per-chain flow watchdog. A high-activity pool that goes silent for + // 10 minutes means the subscription is dead even if the connection + // looks healthy (pongs + other subs satisfy the read deadline). On + // trip: purge the frozen gauges for that chain so the bench page + // never shows a stale value, then force a reconnect by closing the + // connection (the read loop errors out and the outer loop redials). + var lastEventMu sync.Mutex + lastEventByChain := make(map[string]time.Time) + firstEventBySub := make(map[string]bool) + for _, pool := range headLagPools { + lastEventByChain[pool.ChainName] = time.Now() + } + const flowSilence = 10 * time.Minute + watchdogDone := make(chan struct{}) + go func() { + t := time.NewTicker(time.Minute) + defer t.Stop() + for { + select { + case <-watchdogDone: + return + case <-t.C: + lastEventMu.Lock() + tripped := "" + for chain, last := range lastEventByChain { + if time.Since(last) > flowSilence { + tripped = chain + break + } + } + lastEventMu.Unlock() + if tripped != "" { + log.Printf("[HEAD-LAG][CODEX] 🪦 %s subscription silent for >%s — purging gauges and forcing reconnect", tripped, flowSilence) + DeleteHeadLagSeries("codex", tripped, config.MonitorRegion) + _ = conn.Close() + return + } + } + } + }() + defer close(watchdogDone) + pingDone := make(chan struct{}) go func() { t := time.NewTicker(20 * time.Second) @@ -564,11 +632,29 @@ func connectAndMonitorCodex(config *Config, stopChan <-chan struct{}) error { continue } + // graphql-transport-ws: the server terminates ONE subscription + // with {"type":"complete","id":...} or {"type":"error",...}. + // Swallowing these (the pre-2026-07-09 behaviour) leaves a dead + // subscription on a live connection: no read timeout (other + // subs keep the socket busy), frozen gauges, blind alerts. + // Codex killed the Solana sub exactly this way. Force a full + // reconnect + resubscribe instead. + if wsMsg.Type == "complete" || wsMsg.Type == "error" { + return fmt.Errorf("subscription %q terminated by server (type=%s)", wsMsg.ID, wsMsg.Type) + } + // Skip non-data messages if wsMsg.Type != "next" || wsMsg.Payload == nil { continue } + // One-shot log per subscription id: tells us whether the legacy + // unconfirmed sub, the Processed sub, or both deliver on this tier. + if !firstEventBySub[wsMsg.ID] { + firstEventBySub[wsMsg.ID] = true + log.Printf("[HEAD-LAG][CODEX] 📬 First event on subscription %q", wsMsg.ID) + } + // Parse event data payloadBytes, _ := json.Marshal(wsMsg.Payload) var eventData CodexEventData @@ -611,6 +697,10 @@ func connectAndMonitorCodex(config *Config, stopChan <-chan struct{}) error { // Get chain name chainName := getChainNameFromNetworkID(networkID) + lastEventMu.Lock() + lastEventByChain[chainName] = time.Now() + lastEventMu.Unlock() + // Record metrics with tx hash RecordHeadLag("codex", chainName, lagMs, lagSeconds, config.MonitorRegion, event.TransactionHash) RecordCodexBlockNumber(chainName, event.BlockNumber, config.MonitorRegion) @@ -645,6 +735,8 @@ func getChainNameFromNetworkID(networkID int) string { return "base" case 56: return "bnb" + case 4663: + return "robinhood" default: return fmt.Sprintf("network_%d", networkID) } diff --git a/harnesses/aggregator-head-lag/cmd/script/metrics.go b/harnesses/aggregator-head-lag/cmd/script/metrics.go index b31e2c04..55b14b40 100644 --- a/harnesses/aggregator-head-lag/cmd/script/metrics.go +++ b/harnesses/aggregator-head-lag/cmd/script/metrics.go @@ -4,7 +4,6 @@ import ( "fmt" "net/http" "sync" - "github.com/prometheus/client_golang/prometheus" "github.com/prometheus/client_golang/prometheus/promhttp" ) @@ -50,6 +49,10 @@ var ( mobulaLastTxHash *prometheus.GaugeVec mobulaLastTxMu sync.Mutex mobulaLastTxSeen = make(map[string]string) // key: "chain:pool_address" -> last tx_hash + + // WebSocket connection lifecycle (deco/reco visibility in Grafana/Prom) + wsReconnects *prometheus.CounterVec + wsConnected *prometheus.GaugeVec ) func init() { @@ -263,6 +266,46 @@ func init() { []string{"aggregator", "chain", "region", "pool_address", "tx_hash"}, ) prometheus.MustRegister(mobulaLastTxHash) + + wsReconnects = prometheus.NewCounterVec( + prometheus.CounterOpts{ + Name: "ws_reconnects_total", + Help: "Total number of WebSocket reconnections per aggregator (increments on every disconnect, whatever the cause)", + }, + []string{"aggregator", "region"}, + ) + prometheus.MustRegister(wsReconnects) + + wsConnected = prometheus.NewGaugeVec( + prometheus.GaugeOpts{ + Name: "ws_connected", + Help: "WebSocket connection state per aggregator (1 = connected, 0 = disconnected)", + }, + []string{"aggregator", "region"}, + ) + prometheus.MustRegister(wsConnected) +} + +// RecordWSReconnect increments the reconnect counter for an aggregator's WebSocket. +func RecordWSReconnect(aggregator string, region string) { + wsReconnects.WithLabelValues(aggregator, region).Inc() +} + +// DeleteHeadLagSeries removes the head-lag gauges for one (aggregator, +// chain, region) so a dead subscription cannot keep publishing its last +// value (frozen-gauge trap, Codex Solana 2026-07-09). +func DeleteHeadLagSeries(aggregator, chain, region string) { + headLagSeconds.DeleteLabelValues(aggregator, chain, region) + headLagBlocks.DeleteLabelValues(aggregator, chain, region) +} + +// RecordWSConnected sets the connection state gauge for an aggregator's WebSocket. +func RecordWSConnected(aggregator string, region string, connected bool) { + v := 0.0 + if connected { + v = 1.0 + } + wsConnected.WithLabelValues(aggregator, region).Set(v) } // RecordMobulaLastTx overwrites the single live series for (chain, pool_address) @@ -340,8 +383,12 @@ func RecordMetadataLatency(provider string, chain string, latencyMs float64, reg // RecordHeadLag records the head lag for an aggregator on a specific chain func RecordHeadLag(aggregator string, chain string, lagBlocks int64, lagSeconds float64, region string, txHash string) { - // Filter out aberrant values (> 30s likely means connection issues, not real lag) - if lagSeconds < 0 || lagSeconds > 30 { + // Filter out aberrant values. 120s matches the fast-trade filter: it + // keeps out reconnect replay backlogs (minutes-old events redelivered + // after a resubscribe) while letting real high lag through. The old 30s + // cap silently dropped every codex sample on Robinhood Chain (~45s real + // lag), showing "no data" instead of the honest bad number. + if lagSeconds < 0 || lagSeconds > 120 { return } @@ -411,11 +458,11 @@ func StartMetricsServer(addr string) error { // Prometheus metrics endpoint mux.Handle("/metrics", promhttp.Handler()) - mux.Handle("/logs", logsHandler()) // Admin cleanup endpoint setupCleanupEndpoint(mux) // Debug: tail of in-memory log ring (shared loghub package) + mux.Handle("/logs", logsHandler()) return http.ListenAndServe(addr, mux) } diff --git a/harnesses/aggregator-head-lag/cmd/script/mobula_fast_trade_monitor.go b/harnesses/aggregator-head-lag/cmd/script/mobula_fast_trade_monitor.go index 9629ef2e..d45e0ef6 100644 --- a/harnesses/aggregator-head-lag/cmd/script/mobula_fast_trade_monitor.go +++ b/harnesses/aggregator-head-lag/cmd/script/mobula_fast_trade_monitor.go @@ -45,6 +45,11 @@ var fastTradePools = []struct { Address: "0x58f876857a02d6762e0101bb5c46a8c1ed44dc16", // WBNB/BUSD PancakeSwap ChainName: "bnb", }, + { + Blockchain: "evm:4663", + Address: "0x69bfaf19c9f377bb306a89aed9f6b07e2c1a8d9a", // USDG/WETH Robinhood + ChainName: "robinhood", + }, } func runMobulaFastTradeMonitor(config *Config, stopChan <-chan struct{}) { @@ -219,6 +224,8 @@ func getChainNameFromBlockchainFastTrade(blockchain string) string { return "base" case "BNB Smart Chain (BEP20)", "BSC", "evm:56": return "bnb" + case "Robinhood Chain", "evm:4663": + return "robinhood" default: return blockchain } diff --git a/infrastructure/monitoring/prometheus/alert_rules.yml b/infrastructure/monitoring/prometheus/alert_rules.yml index 233c7354..f06632b1 100644 --- a/infrastructure/monitoring/prometheus/alert_rules.yml +++ b/infrastructure/monitoring/prometheus/alert_rules.yml @@ -92,7 +92,14 @@ groups: # (e.g. Codex WS disconnect) while others keep running — global absent() # would NOT catch this. - alert: AggregatorHeadLagStale - expr: (time() - timestamp(head_lag_seconds)) > 300 + # changes() et non timestamp(): un gauge gelé (souscription morte, + # harness qui ré-expose la dernière valeur) a des samples frais à + # chaque scrape — timestamp() ne voit rien (Codex Solana 2026-07-09, + # 8h de 26.3s figés sans alerte). Un vrai flux a toujours du jitter. + # Scope mobula uniquement: un feed tiers (codex/gecko) qui meurt est + # leur histoire, le watchdog purge ses gauges et la page montre le + # trou. Pas de bruit Slack pour eux. + expr: changes(head_lag_seconds{aggregator="mobula"}[30m]) == 0 for: 1m labels: severity: warning @@ -156,7 +163,8 @@ groups: # Aggregator WS silently down: connection-state gauge from the # reconnect instrumentation (PR #950). - alert: AggregatorWSDisconnected - expr: ws_connected == 0 + # mobula only, même logique que AggregatorHeadLagStale. + expr: ws_connected{aggregator="mobula"} == 0 for: 10m labels: severity: warning diff --git a/public/logos/robinhood.png b/public/logos/robinhood.png new file mode 100644 index 00000000..2746f39b Binary files /dev/null and b/public/logos/robinhood.png differ diff --git a/src/lib/logo-manifest.ts b/src/lib/logo-manifest.ts index 433f4e77..d580c026 100644 --- a/src/lib/logo-manifest.ts +++ b/src/lib/logo-manifest.ts @@ -43,6 +43,7 @@ const RAW: Record = { sonic: "/logos/sonic.png", monad: "/logos/monad.png", megaeth: "/logos/megaeth.png", + robinhood: "/logos/robinhood.png", onfinality: "/logos/onfinality.png", berachain: "/logos/berachain.png",