feat(telemetry): sharded compact-binary CXO telemetry leaves (fillable busy-hub Root) - #4190
Merged
Merged
Conversation
…e busy-hub Root) Replace the N per-transport transports/<uuid>/current JSON telemetry leaves with 16 fixed sharded binary leaves at transports/telemetry/<sh> (shard = uuid[0]>>4). A busy hub's ~851 per-transport leaves (~1700 CXO objects) collapse to a handful, so TPD's whole-Root fill completes over the short announce conn instead of undercounting. The byte layout lives in a single shared codec (pkg/telemetrywire) imported by both the visor publisher (pkg/visor/stats) and the TPD reader (pkg/deployment/tpd/cxoaggregator) so the two sides can't drift. Version byte 0x02 gates the format; entries are 53 bytes and carry the transport's passively-observed peak-goodput throughput estimate (GetThroughputBps) alongside bandwidth/latency, which TPD records per-transport. TPD reads the shards and applies each row exactly as a current snapshot; it keeps the old transports/<uuid>/current path as a fallback for not-yet-upgraded visors, preferring shards when a Root carries them. Deploy is TPD-FIRST: TPD (shards + old-current fallback) must ship before visors switch to shard-only publishing. Also re-consolidates the tp-list discovery leaf onto this telemetry feed (one visor<->TPD CXO connection), redoing the reverted skycoin#4184 correctly: the now-small sharded Root is what makes the single combined feed fillable. The dedicated port-69 feed remains available behind Stats.DedicatedTPListFeed as a transitional escape hatch.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Replace the N per-transport
transports/<uuid>/currentJSON telemetry leaves on the visor→TPD CXO telemetry feed (dmsg port 50) with 16 fixed sharded binary leaves attransports/telemetry/<sh>(shard =uuid[0]>>4). A busy hub's ~851 per-transport leaves (~1700 CXO objects) collapse to ≤16 leaves, so TPD's whole-Root fill finishes over its short-lived announce conn instead of undercounting busy-hub telemetry.Shared codec (no drift)
The byte layout lives in one shared package,
pkg/telemetrywire, imported by both the visor publisher (pkg/visor/stats) and the TPD reader (pkg/deployment/tpd/cxoaggregator). Version byte0x02gates the format;DecodeShardvalidates version/shard-range/length strictly. Entries are 53 bytes and carry the transport's passively-observed peak-goodput throughput estimate (ManagedTransport.GetThroughputBps) alongside sent/recv bytes and latency — a capacity metric not derivable from the cumulative counters. TPD records it per-transport (peak-preserving) and overlays it ontoEntry.ThroughputBps.TPD fallback
TPD reads the shard leaves and applies each row exactly as a
currentsnapshot (bandwidth, throughput, latency, per-type uptime heartbeat). It keeps the legacytransports/<uuid>/currentpath as a fallback for not-yet-upgraded visors, preferring shards when a Root carries any. Deploy is TPD-FIRST: TPD (shards + old-current fallback) must ship before visors switch to shard-only publishing.tp-list re-consolidation (redo of reverted #4184)
Also re-consolidates the tp-list discovery leaf back onto this telemetry feed so a visor holds one CXO connection to TPD, not two. This is the change reverted in #4189 — safe now because the sharded telemetry is exactly what keeps the combined Root small enough to fill. The dedicated port-69 feed stays available behind
Stats.DedicatedTPListFeedas a transitional escape hatch (default off = consolidated).Tests
New codec round-trip/rejection tests; publish-shape tests proving 800 transports publish ≤16 leaves (hydrate and sampler); shard change-gate (idle shard not re-Put); TPD shard-reconcile + old-current fallback + shard-preference.
go build,go vet, andgo testgreen across the codec,pkg/visor/stats,pkg/visor, and the TPD aggregator/store packages.