s0racle-observer

A lightweight Rust daemon that joins Solana's gossip layer, probes every validator TPU endpoint via QUIC, and publishes signed network-health attestations on-chain every 10 seconds. It is the off-chain data-collection arm of the s0racle protocol — the piece that feeds real, ground-truth measurements into the s0racle-program smart contract.

---

Table of Contents

1. Why This Exists
2. How It Works — End to End
3. Program Identity & Contract Address
4. Architecture — Module Map
5. The Probe Cycle — Step by Step
6. Metrics Collected & Published
7. Gossip Layer Deep Dive
8. QUIC Prober Deep Dive
9. Stake-Weighted Reachability
10. Client Diversity Tracking
11. Slot Latency Measurement
12. On-Chain Attestation Submission
13. Auto-Registration Logic
14. Retry & Fault Tolerance
15. Configuration Reference
16. Environment Variable Overrides
17. Key Features
18. Prerequisites
19. Build & Run
20. Multi-Region Deployment
21. Structured Logging
22. Project Structure
23. Dependency Versions

---

Why This Exists

Solana's validator set is spread across dozens of countries, runs multiple competing client implementations, and communicates entirely over QUIC. Today, there is no trust-minimised way to answer:

• Right now, what % of validators can actually accept transactions via QUIC?
• Is slot propagation slower in Asia than in the EU at this moment?
• Which validator client — Agave, Firedancer, or Jito — dominates the active set?
• What % of staked SOL is behind validators that are reachable?

Existing monitoring tools answer these questions by polling off-chain APIs — a single point of failure, invisible to on-chain programs, and trivially censorable.

s0racle-observer fixes this by running inside the gossip layer itself — the same transport validators use to share cluster state. Because it participates in gossip as a real node, it sees ground-truth peer data, not a sanitised API response. Every measurement it produces is signed by an on-chain identity and published to a Solana program where any dApp or smart contract can read it.

---

How It Works — End to End

┌─────────────────────────────────────────────────────────┐
│                    s0racle-observer                      │
│                                                         │
│  1. Join Gossip ──► CRDS table fills with peer data     │
│                     (TPU addresses, client versions,     │
│                      epoch slots, shred versions)        │
│                                                         │
│  2. Every 2s ──────► Refresh TPU peer list from CRDS    │
│                      Track highest observed gossip slot  │
│                                                         │
│  3. Every 10s ─────► Fire QUIC probes → all TPU peers   │
│                      Collect RTTs + reachable pubkeys    │
│                      ──────────────────────────────────  │
│                      Read client versions from CRDS      │
│                      ──────────────────────────────────  │
│                      Fetch vote accounts → stake map     │
│                      Compute stake-weighted reachability │
│                      ──────────────────────────────────  │
│                      Fetch cluster slot from RPC         │
│                      Compute slot lag in ms              │
│                      ──────────────────────────────────  │
│                      Submit all data as on-chain         │
│                      attestation (with 3x retry)         │
└─────────────────────────────────────────────────────────┘
           │
           ▼
  s0racle-program (Solana)
  NetworkHealthAccount ──► Any dApp / smart contract reads

---

Program Identity & Contract Address

The observer submits attestations to the s0racle-program on-chain. The default program ID is set in Config.toml:

Network	Program ID
Devnet	2paXpX8Ze3tvYezviSwQJSSihG3LbrDiD7SNsaFwgTow
Mainnet-beta	not yet deployed

The observer derives three PDAs locally (no lookup needed) to construct each transaction:

PDA	Seeds	Purpose
observer_account	[b"observer", observer_wallet_pubkey]	The observer's on-chain identity and latest attestation
network_health	[b"network_health"]	The global oracle output account — written by every attestation
registry	[b"registry"]	Protocol config — checked for paused flag before every submission

---

Architecture — Module Map

src/
├── main.rs          — Entry point: event loop, wiring, graceful shutdown
├── config.rs        — TOML config loading + environment variable overrides
├── gossip.rs        — Gossip node lifecycle, CRDS reads, slot tracking, client diversity
├── prober.rs        — Concurrent QUIC probes, RTT calculation, stake weighting
└── attestation.rs   — On-chain registration + attestation submission via anchor-client

Concurrency model

The daemon runs entirely inside a single Tokio async runtime (#[tokio::main]). The two timers — peer refresh (every 2 s) and attestation (every 10 s) — are driven by tokio::select! on tokio::time::interval tickers, both with MissedTickBehavior::Skip so a slow probe round never queues duplicate attestations.

The QUIC probe fan-out uses tokio::task::JoinSet — one task per TPU peer, all running concurrently. Blocking operations (RPC calls, on-chain submissions) are dispatched to tokio::task::spawn_blocking to avoid stalling the async executor.

The gossip service runs in its own background threads (managed by GossipService) and communicates with the observer through the thread-safe Arc<ClusterInfo> CRDS table.

---

The Probe Cycle — Step by Step

Peer refresh (every 2 seconds)

cluster_info.all_peers()
  → filter out self
  → keep only peers with a QUIC TPU address
  → store as Vec<(Pubkey, SocketAddr)>

update_observed_slot()
  → scan new EpochSlots CRDS entries since last cursor position
  → update highest_observed_slot

The peer list is kept fresh at 2-second intervals so the 10-second probe always uses the latest gossip view. Log line emitted whenever peer count changes.

Attestation cycle (every 10 seconds)

1. Guard: skip if tpu_peers is empty (gossip hasn't filled yet)

2. prober::probe_validator(tpu_peers, max_probe_targets)
   → fan-out QUIC handshakes (1500ms timeout per peer)
   → collect ProbeResult { tpu_reachable, tpu_probed, avg_rtt_us, p95_rtt_us, reachable_pubkeys }

3. gossip::compute_client_diversity(cluster_info)
   → read Version CRDS for every peer
   → bucket into { agave, firedancer, jito, solana_labs, other, unknown }

4. [spawn_blocking]
   a. gossip::compute_slot_latency_ms(rpc, observed_slot)
      → rpc.get_slot() - highest_observed_slot → × 400ms/slot
      → clamped to MAX_SLOT_LATENCY_MS (10,000ms)

   b. prober::fetch_stake_map(rpc)
      → rpc.get_vote_accounts() → HashMap<Pubkey, u64>

   c. prober::compute_stake_pct(reachable_pubkeys, stake_map)
      → reached_stake / total_stake × 100 → u8

   d. attestation::submit_attestation(...)
      → build + send SubmitAttestation tx (3 retries, 500ms backoff)

---

Metrics Collected & Published

Every 10-second probe round produces the following metrics, all submitted to the on-chain submit_attestation instruction:

Metric	Source	Type	Description
tpu_reachable	QUIC prober	u16	Number of validators that completed a QUIC handshake within 1500ms
tpu_probed	QUIC prober	u16	Total validators attempted this round (capped at max_probe_targets)
avg_rtt_us	QUIC prober	u32	Mean QUIC handshake round-trip time in microseconds
p95_rtt_us	QUIC prober	u32	95th-percentile QUIC RTT in microseconds — more robust than mean for latency outliers
slot_latency_ms	Gossip + RPC	u32	How far this observer lags behind the cluster in milliseconds = (cluster_slot - observed_slot) × 400
agave_count	Gossip CRDS	u16	Validators running the Agave client
firedancer_count	Gossip CRDS	u16	Validators running Firedancer
jito_count	Gossip CRDS	u16	Validators running Jito
solana_labs_count	Gossip CRDS	u16	Validators running the original Solana Labs client
other_count	Gossip CRDS	u16	Validators with unrecognised or unpublished client (other + unknown merged)
reachable_stake_pct	RPC + prober	u8	% of total activated stake (lamports) held by validators that responded to QUIC — the most important metric

Why reachable_stake_pct matters most

Reaching 100 low-stake validators while 20 high-stake validators are unreachable gives a misleading raw reachability %. Stake-weighting corrects for this — if validators holding 80% of all staked SOL are QUIC-reachable, the network can confirm transactions even if many small validators are down. This metric is the real health signal for transaction landing rates.

---

Gossip Layer Deep Dive

File: src/gossip.rs

What is Solana gossip?

Solana's gossip protocol is a peer-to-peer data sharing layer where every node (validators, RPCs, observers) exchanges "CRDS" (Cluster Replicated Data Store) records. These records include:

• ContactInfo — node identity, addresses (TPU, gossip, RPC)
• Version — which client software and version the node runs
• EpochSlots — a compact bitset of which slots the node has seen

The observer joins this gossip network as a non-voting, non-producing participant. It has a real node identity (keypair) but does not produce blocks or cast votes. This gives it read access to the same CRDS table that validators use to find each other.

Gossip startup sequence

// 1. Fetch the cluster's shred_version from any RPC node
//    (nodes with wrong shred_version are silently dropped by peers)
let shred_version = rpc.get_cluster_nodes()...;

// 2. Build a ContactInfo with our keypair and the correct shred_version
let contact_info = ContactInfo::new(keypair.pubkey(), timestamp(), shred_version);

// 3. Create a ClusterInfo — the in-process CRDS database
let cluster_info = Arc::new(ClusterInfo::new(contact_info, keypair, SocketAddrSpace::new(true)));

// 4. Point to the network's entrypoint (the door in)
cluster_info.set_entrypoint(ContactInfo::new_gossip_entry_point(&entrypoint_addr));

// 5. Set our advertised gossip address
cluster_info.set_gossip_socket(gossip_addr);

// 6. Bind a UDP socket and start the GossipService background threads
let gossip_service = GossipService::new(&cluster_info, None, udp_socket, ...);

Once started, GossipService runs push/pull/ping/pong cycles in the background, continuously filling the CRDS table. The observer reads from it without locking — ClusterInfo::all_peers() and ClusterInfo::get_node_version() are both thread-safe.

Shred version — why it matters

The shred version is a cluster-wide discriminator that prevents nodes from different networks (mainnet, devnet, testnet) from contaminating each other's gossip tables. The observer fetches it dynamically at startup from getClusterNodes — hardcoding it would break across network resets.

EpochSlots and slot tracking

pub fn update_observed_slot(cluster_info: &ClusterInfo, cursor: &mut Cursor, highest: &mut u64) {
    let epoch_slots = cluster_info.get_epoch_slots(cursor);
    for es in epoch_slots.iter() {
        let Some(min_slot) = es.first_slot() else { continue; };
        for slot in es.to_slots(min_slot) {
            if slot > *highest { *highest = slot; }
        }
    }
}

The Cursor tracks how far through the CRDS log we've already read — each call returns only new entries since the last poll. The highest slot ever observed via gossip is used for the slot latency calculation.

---

QUIC Prober Deep Dive

File: src/prober.rs

Why QUIC, not RPC?

Solana validators accept transactions over QUIC TPU (Transaction Processing Unit) endpoints, not over RPC. A validator that is reachable via RPC might still have a broken or blocked QUIC TPU port — meaning transactions can't land even though monitoring looks fine. The observer probes QUIC directly, testing the actual transaction submission path.

Probe mechanics

// TLS: custom verifier that skips certificate validation
// This is correct — Solana validators use self-signed certs;
// the identity check is done at the Solana protocol level via Ed25519
tls_config.alpn_protocols = vec![b"solana-tpu".to_vec()];

// QUIC endpoint bound to a random local port (0.0.0.0:0)
let endpoint = Endpoint::client("0.0.0.0:0".parse()?)?;
endpoint.set_default_client_config(...);

// Fan-out: one JoinSet task per peer, all in parallel
for (pubkey, addr) in peers.take(max_targets) {
    join_set.spawn(async move {
        tokio::time::timeout(
            Duration::from_millis(1500),  // 1.5-second handshake timeout
            probe_socket(&endpoint, addr)
        ).await
    });
}

Each probe task:

1. Calls endpoint.connect(addr, "validator") — initiates the QUIC handshake
2. Awaits the connection — measures time from Instant::now() to handshake completion
3. Immediately closes the connection with reason code 0 and message b"done" — we only need the handshake RTT
4. Returns the RTT in microseconds, or an error on timeout/failure

RTT calculation

// Average
avg_rtt_us = rtts.iter().map(|&r| r as u64).sum::<u64>() / rtts.len() as u64;

// P95 (sorted, 95th-percentile index)
rtts.sort_unstable();
let idx = ((rtts.len() - 1) as f64 * 0.95) as usize;
p95_rtt_us = rtts[idx];

P95 is more useful than mean for latency analysis — a small number of very slow validators can inflate the mean significantly while P95 gives a truer picture of the "typical worst case".

TLS certificate verification skip

The SkipServerVerification struct implements rustls::client::danger::ServerCertVerifier and accepts any certificate. This is intentional — Solana's QUIC endpoints use self-signed certificates; authentication happens at the application layer via Ed25519 signatures on transactions. The supported signature schemes include ED25519, ECDSA_NISTP256_SHA256, and RSA_PSS_SHA256.

max_probe_targets

The peer list is truncated to max_probe_targets (default: 200) before probing. This caps the burst of outgoing QUIC connections per round, preventing the observer from overwhelming its own NIC or getting rate-limited. At 200 targets with a 1500ms timeout, the probe round completes within roughly 2–4 seconds depending on network conditions.

---

Stake-Weighted Reachability

File: src/prober.rs — fetch_stake_map + compute_stake_pct

Fetching the stake map

pub fn fetch_stake_map(rpc: &RpcClient) -> HashMap<Pubkey, u64> {
    let status = rpc.get_vote_accounts()?;
    // Include BOTH current and delinquent — delinquent still hold activated stake
    for acc in status.current.iter().chain(status.delinquent.iter()) {
        map.entry(pk)
           .and_modify(|s| *s = s.saturating_add(acc.activated_stake))
           .or_insert(acc.activated_stake);
    }
}

Note: delinquent validators are included because they still hold staked SOL. A validator that is delinquent (not voting) but QUIC-reachable is still relevant — stake delegated to it still counts for economic security.

Computing the percentage

pub fn compute_stake_pct(reachable: &[Pubkey], stake_map: &HashMap<Pubkey, u64>) -> u8 {
    let total_stake: u64 = stake_map.values().sum();
    let reached_stake: u64 = reachable.iter()
        .filter_map(|pk| stake_map.get(pk))
        .sum();
    ((reached_stake as u128 * 100 / total_stake as u128).min(100)) as u8
}

Uses u128 arithmetic for the multiplication to avoid overflow on large stake totals. The result is clamped to 100 and stored as a u8 (0–100).

---

Client Diversity Tracking

File: src/gossip.rs — compute_client_diversity

pub fn compute_client_diversity(cluster_info: &ClusterInfo) -> ClientCounts {
    for (info, _) in cluster_info.all_peers() {
        match cluster_info.get_node_version(info.pubkey()) {
            Some(v) => match v.client() {
                ClientId::Agave      => counts.agave += 1,
                ClientId::Firedancer => counts.firedancer += 1,
                ClientId::JitoLabs  => counts.jito += 1,
                ClientId::SolanaLabs => counts.solana_labs += 1,
                ClientId::Unknown(_) => counts.other += 1,
            },
            None => counts.unknown += 1,  // peer hasn't published Version CRDS yet
        }
    }
}

This is a pure in-memory read of the CRDS table — no extra RPC call, no network traffic. The unknown count (peers without a published Version record) is merged into other_count when submitted on-chain.

Why client diversity matters

• Monoculture risk: If 95% of stake runs one client, a single bug in that client can halt the network.
• Fork risk: Client diversity affects how forks resolve — different clients may process certain edge-case transactions differently.
• Upgrade tracking: Watching firedancer_count grow over time shows adoption of new clients.

---

Slot Latency Measurement

File: src/gossip.rs — compute_slot_latency_ms

const SLOT_DURATION_MS: u64 = 400;  // Solana's ~400ms target slot time

pub fn compute_slot_latency_ms(rpc: &RpcClient, observed_slot: u64) -> u32 {
    let cluster_slot = rpc.get_slot()?;
    let lag_slots = cluster_slot.saturating_sub(observed_slot);
    lag_slots.saturating_mul(SLOT_DURATION_MS).min(u32::MAX as u64) as u32
}

What it measures: The difference between the global cluster slot (fetched from RPC — this is the network's agreed-upon current slot) and the highest slot this observer has ever seen announced via gossip. If the observer is well-connected, this difference should be 0–2 slots (0–800ms). A lag of 10+ slots (4+ seconds) indicates the observer is poorly connected or the network is experiencing propagation issues.

Why it's useful across regions: cluster_slot is the same for all observers (it's the canonical chain slot). Only observed_slot varies by geography — observers in regions with poor peering see slots later. Comparing slot_latency_ms across regions reveals where block propagation is slowest.

Clamping: Raw latency is clamped to MAX_SLOT_LATENCY_MS (10,000ms = 10 seconds) before submission, matching the on-chain program's accepted range.

---

On-Chain Attestation Submission

File: src/attestation.rs — submit_attestation

// 1. Parse program_id from config string
// 2. Build anchor-client Client with confirmed commitment
// 3. Derive the three PDAs (observer, network_health, registry)
// 4. Check registry.paused — skip the round if true (no wasted tx fee)
// 5. Build + send the SubmitAttestation instruction with all 11 fields
// 6. On success: log the transaction signature
// 7. On failure: retry up to 3 times with 500ms sleep between attempts

Transaction structure

The instruction accounts required:

Account	Role
authority	Signer — observer's wallet (pays tx fee)
observer_account	Mutable PDA — updated with the new attestation
network_health	Mutable PDA — global oracle, updated immediately
registry	Read-only PDA — checked for paused state in-program
clock	Sysvar — provides current slot and timestamp

Paused-registry check (client-side)

Before building the transaction, the observer fetches RegistryAccount and checks registry.paused. If true it skips the round immediately — no transaction fee is spent. The on-chain program also enforces this as a constraint, so a race condition (registry paused between the client check and the on-chain execution) results in an error that the retry logic handles.

---

Auto-Registration Logic

File: src/attestation.rs — register_observer_if_needed

On first startup, the observer checks whether its ObserverAccount PDA already exists on-chain:

if program.rpc().get_account(&observer_pda).is_ok() {
    tracing::info!("observer already registered: {}", observer_pda);
    return Ok(());
}

If the account does not exist, it sends a RegisterObserver instruction:

• Transfers min_stake_lamports (read from RegistryAccount) from the observer's wallet to the PDA as escrow
• Initialises the ObserverAccount with the configured region, authority, and stake
• Increments the registry's observer_count and active_count

If registration fails (network error, insufficient balance, registry paused, registry full), the daemon logs a warning and enters probe-only mode — it continues probing and collecting metrics but does not submit attestations on-chain. This prevents a crash-loop on transient failures.

Registration uses the same 3-retry / 2-second-backoff pattern as attestation submission.

---

Retry & Fault Tolerance

Failure point	Behaviour
Registration fails	Warn, enter probe-only mode, daemon keeps running
Probe round has zero peers	Skip attestation, log warning, retry next tick
QUIC handshake times out (1500ms)	Log warning per peer, count as unreachable
getVoteAccounts RPC fails	stake_map returns empty; reachable_stake_pct = 0
getSlot RPC fails	slot_latency_ms = 0 (rather than crash)
Attestation tx fails	Retry up to 3×, 500ms sleep between; log error if all fail
Gossip service thread panics	Logged at error level; daemon continues
Ctrl+C received	Sets AtomicBool exit = true, gossip service joins cleanly

All arithmetic uses saturating operations throughout — no integer overflow panics are possible.

---

Configuration Reference

Copy Config.toml.example to Config.toml and fill in your values:

# Geographic region this observer node is located in.
# Must be one of: Asia, US, EU, SouthAmerica, Africa, Oceania, Other
region = "Asia"

# Path to the Solana keypair JSON file used as the observer's identity.
# This wallet pays transaction fees and is the on-chain registered observer.
keypair_path = "./observer-keypair.json"

# Solana RPC endpoint. Used for:
#   - Fetching shred_version at startup
#   - get_vote_accounts (stake map)
#   - get_slot (slot latency)
#   - Submitting transactions via anchor-client
rpc_url = "https://api.devnet.solana.com"

# Gossip entrypoint — the door into the Solana gossip network.
# Devnet:   entrypoint.devnet.solana.com:8001
# Mainnet:  entrypoint.mainnet-beta.solana.com:8001
gossip_entrypoint = "entrypoint.devnet.solana.com:8001"

# The s0racle-program program ID on-chain.
program_id = "2paXpX8Ze3tvYezviSwQJSSihG3LbrDiD7SNsaFwgTow"

# How often to run a probe round and submit an attestation.
# Default: 10 seconds. The on-chain program expects attestations per ~10s.
attestation_interval_secs = 10

# Maximum number of TPU peers to probe per round.
# Higher = more coverage, more outgoing connections.
# Default: 200 (covers a large fraction of the validator set).
max_probe_targets = 200

# The gossip address this node ADVERTISES to peers.
# Must be a routable address (not 0.0.0.0) if you want peers to reach back.
# For a simple observer that only reads gossip, 0.0.0.0:8001 is fine.
gossip_addr = "0.0.0.0:8001"

# The address this node BINDS its gossip UDP socket to.
# Typically matches gossip_addr or 0.0.0.0:<port>.
gossip_host = "0.0.0.0:8001"

---

Environment Variable Overrides

Every config field can be overridden with an environment variable — the same binary deploys across all regions with no config file changes needed.

Environment Variable	Config field	Example
SORACLE_REGION	region	EU
SORACLE_KEYPAIR_PATH	keypair_path	/secrets/observer.json
SORACLE_RPC_URL	rpc_url	https://my-rpc.example.com
SORACLE_GOSSIP_ENTRYPOINT	gossip_entrypoint	entrypoint.mainnet-beta.solana.com:8001
SORACLE_GOSSIP_ADDR	gossip_addr	203.0.113.42:8001
SORACLE_GOSSIP_HOST	gossip_host	0.0.0.0:8001
SORACLE_PROGRAM_ID	program_id	2paXpX8Ze3tvYezviSwQJSSihG3LbrDiD7SNsaFwgTow
SORACLE_CONFIG	config file path	/etc/soracle/Config.toml

Invalid SORACLE_REGION values fall back to Other with a warning log.

---

Key Features

Inside the gossip layer. The observer participates in Solana's P2P gossip protocol as a real node — not as an RPC client. It receives the same peer data validators share with each other, including TPU addresses and client version records that are not exposed through public RPC APIs.

Concurrent QUIC probing. All TPU peer probes in a round fire simultaneously using Tokio's JoinSet. A round of 200 probes completes in roughly the time of the slowest response (1500ms max), not 200 × 1500ms. This is the only way to probe the full validator set within a 10-second window.

Stake-weighted reachability. Raw reachability counts (50 validators reached out of 60 probed) can be misleading. The observer fetches getVoteAccounts to build a stake map and computes what fraction of total staked SOL is behind reachable validators — the metric that actually matters for transaction landing rates.

Zero-cost client diversity. Client version information is read directly from the CRDS table — no extra RPC calls, no network traffic, no latency. The Version records are already in memory from gossip.

Slot-based latency. Rather than measuring artificial ping round-trips, the observer measures how far it lags behind the cluster's canonical slot — a direct measure of block propagation health from its geographic vantage point.

Auto-registration. First-run setup is fully automatic. The observer checks for its on-chain PDA, registers if needed, and proceeds to probe. No manual on-chain transaction is required.

Graceful pause handling. The observer checks RegistryAccount.paused before every submission — both client-side (to avoid spending fees) and the on-chain program enforces it as a constraint. During a pause, the observer keeps running and probing; it just doesn't submit.

Probe-only fallback. If registration fails for any reason (insufficient funds, full registry, paused, network error), the daemon enters probe-only mode rather than crashing. This keeps the gossip participant alive and ready to resume submitting once the issue is resolved.

Env-var driven deployment. A single compiled binary runs everywhere. Region, RPC endpoint, keypair path, and all other settings are overridable via environment variables — ideal for Kubernetes, Docker, or systemd deployments.

Structured logging. All output is structured JSON-compatible via tracing with configurable log levels via RUST_LOG. Gossip and metrics noise is suppressed by default (solana_gossip=warn, solana_metrics=off).

---

Prerequisites

Tool	Version	Notes
Rust	stable (2024 edition)	rustup install stable
Solana CLI	≥ 2.x	Required to generate keypair and airdrop
aws-lc-rs build deps	—	On Ubuntu: sudo apt install cmake clang
s0racle-program	same repo	Linked as a local path dependency: ../s0racle-program/programs/s0racle-program

The binary links against aws-lc-rs for TLS (used by rustls). This requires cmake and a C compiler at build time. On macOS, Xcode command-line tools provide these automatically.

---

Build & Run

1. Generate an observer keypair

solana-keygen new -o observer-keypair.json

Note the public key — this becomes your observer's on-chain identity.

2. Fund the keypair

The observer needs SOL to:

• Pay the min_stake_lamports escrow on registration (set by the protocol admin, currently on devnet)
• Pay transaction fees for every attestation (~0.000005 SOL each, ×6/minute = ~0.0018 SOL/hour)

# Devnet airdrop
solana airdrop 2 $(solana-keygen pubkey observer-keypair.json) --url devnet

3. Configure

cp Config.toml.example Config.toml
# Edit Config.toml: set region, keypair_path, rpc_url, gossip_entrypoint

4. Build

# Requires s0racle-program to be at ../s0racle-program/programs/s0racle-program
cargo build --release

5. Run

cargo run --release

# or with a custom config path
SORACLE_CONFIG=/etc/soracle/Config.toml cargo run --release

# or with env overrides
SORACLE_REGION=EU SORACLE_RPC_URL=https://my-rpc.example.com cargo run --release

First-run output (expected)

INFO  starting sOracle observer daemon
INFO  config loaded: Config { region: Asia, ... }
INFO  keypair loaded
INFO  fetched shred version: 50093
INFO  Gossip started - node id: 7xKXtg2CW87d97TXJSDpbD5jBkheTqA83TZRuJosgAsU
INFO  Advertised gossip address: 0.0.0.0:8001
INFO  registering observer on sOracle...
INFO  observer registered on sOracle: 4rMkXTsTa4nHF3HiM9...
INFO  TPU peers updated: 0 → 147
INFO  probe complete — reachable: 134/147 | avg_rtt: 82341us | p95_rtt: 213450us
INFO  client diversity (total 147) — agave: 98 | firedancer: 22 | jito: 18 | labs: 4 | other: 5
INFO  slot latency: 800ms
INFO  stake-weighted reach: 91%
INFO  attestation submitted: 5J3hKmNa...

Subsequent runs

INFO  observer already registered: 9mXkjW2aB...
INFO  TPU peers updated: 0 → 152
... (probe cycles continue)

---

Multi-Region Deployment

Run one observer instance per region using environment variable overrides. No config file changes needed:

Asia (machine A):

SORACLE_REGION=Asia \
SORACLE_KEYPAIR_PATH=/secrets/asia-keypair.json \
SORACLE_GOSSIP_ADDR=203.0.113.10:8001 \
SORACLE_GOSSIP_HOST=0.0.0.0:8001 \
./soracle-observer

EU (machine B):

SORACLE_REGION=EU \
SORACLE_KEYPAIR_PATH=/secrets/eu-keypair.json \
SORACLE_GOSSIP_ADDR=198.51.100.20:8001 \
SORACLE_GOSSIP_HOST=0.0.0.0:8001 \
./soracle-observer

US (machine C):

SORACLE_REGION=US \
SORACLE_KEYPAIR_PATH=/secrets/us-keypair.json \
SORACLE_GOSSIP_ADDR=192.0.2.30:8001 \
SORACLE_GOSSIP_HOST=0.0.0.0:8001 \
./soracle-observer

Each instance needs its own funded keypair. The gossip_addr should be the machine's public IP on the gossip port. Each observer registers independently on-chain and contributes its region's data to the NetworkHealthAccount.

Port requirements

Port	Protocol	Direction	Purpose
8001 (configurable)	UDP	Inbound + Outbound	Solana gossip
Ephemeral	UDP	Outbound	QUIC probes to validator TPU ports

The observer does not expose any HTTP or monitoring port itself — all telemetry is on-chain.

---

Structured Logging

Log levels are controlled via RUST_LOG:

# Default (recommended for production)
RUST_LOG=info cargo run --release

# Verbose gossip debugging
RUST_LOG=debug,solana_gossip=info cargo run --release

# Minimal output
RUST_LOG=warn cargo run --release

Default filter: info,solana_metrics=off,solana_gossip=warn

This suppresses:

• solana_metrics — high-frequency internal metric submissions
• solana_gossip — very verbose per-message gossip logs (degraded to warn)

Key log lines to monitor in production:

Log	Meaning
TPU peers updated: X → Y	Peer count changed — gossip is healthy if Y grows
probe complete — reachable: X/Y	X of Y validators responded to QUIC
stake-weighted reach: X%	Stake-weighted reachability this round
attestation submitted: <sig>	On-chain submission succeeded
attestation failed: ...	All 3 retries failed — check RPC, balance, network
registry paused, skipping	Protocol admin paused submissions
no TPU peers yet, skipping	Gossip not ready yet — normal for first 5–10s
slot latency Xms clamped to Yms	Raw latency exceeded the on-chain max (rare)

---

Project Structure

s0racle-observer-main/
├── Cargo.toml                # Package manifest + all dependencies
├── Cargo.lock                # Locked dependency versions
├── Config.toml.example       # Template config — copy to Config.toml
├── README.md                 # Original short README
└── src/
    ├── main.rs               # Entry point: Tokio runtime, event loop, wiring
    ├── config.rs             # Config struct: TOML parsing + env var overrides
    ├── gossip.rs             # GossipService startup, CRDS reads, slot tracking,
    │                         # client diversity, slot latency computation
    ├── prober.rs             # QUIC fan-out probe, RTT stats, stake map fetch,
    │                         # stake-weighted reachability calculation
    └── attestation.rs        # Auto-registration + SubmitAttestation tx builder

---

Dependency Versions

Crate	Version	Purpose
tokio	1.51.0	Async runtime (full features: timers, tasks, signals)
quinn	0.11.9	QUIC protocol implementation
rustls	0.23.37	TLS 1.3 with aws-lc-rs crypto backend
solana-gossip	3.1.11	ClusterInfo, GossipService, CRDS, ContactInfo
solana-streamer	3.1.11	SocketAddrSpace
solana-version	3.1.11	ClientId enum for client diversity
solana-client	3.1.11	RpcClient — getSlot, getVoteAccounts, getClusterNodes
solana-sdk	3.0.0	Pubkey, Keypair, Signer
anchor-client	0.32	High-level Anchor program interaction
anchor-lang	0.32	Pubkey::find_program_address, instruction types
s0racle-program	local path	Instruction + account types (shared with the on-chain program)
serde	1.0.228	Config struct deserialization (with derive)
toml	1.1.2	TOML config file parsing
tracing	0.1.44	Structured logging macros
tracing-subscriber	0.3.23	Log formatting + RUST_LOG env filter
anyhow	1.0.102	Ergonomic error handling
borsh	1.6.1	Binary serialization (used by Anchor)

---

License

See LICENSE in the root of the repository.

---

Built with Rust · Tokio · Quinn · Solana Gossip · Anchor