Repository navigation
feat(nodes): split Heard By into direct and relay observers (#76) - #451
Conversation
Tests first, red against master: - heard_by_test.go: the per-observation rule (flood empty path, flood last hop by unique prefix, zero-hop DIRECT/TRANSPORT_DIRECT, direct paths never credit, frame header wins over the first-ingested route, mixed mask is unknown), relay observers from every observation, signal only from direct receptions, bulk = single health, hidden observer identities, wireRoute on every observation path, StoreObs layout, OpenAPI schemas, and GetBulkHealth releasing its node rows. - heard_by_bench_test.go: bulk, single and hub health on a 30K transmission / 150K observation fixture. - test-heard-by.js and test-issue-76-heard-by-e2e.js (registered in test-all.sh and the shard manifest); existing fixtures move from observers to directObservers / relayObservers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The node health endpoints listed every observer of a node's traffic with the SNR/RSSI of whichever node that observer last heard. They now return directObservers (observers that received the node's own transmission) and relayObservers (everyone else, without signal values); stats.avgSnr covers direct receptions only. Only an ADVERT shows who transmitted, read per observation (firmware Mesh.cpp): an empty flood path or a zero-hop DIRECT/TRANSPORT_DIRECT path credits the advertiser, a flood path credits its last hop when that hop is the node by the #64 relay evidence (full key or unique wire prefix, never a listener), and a non-empty direct path credits nobody. The route type comes from the observation's own frame header (StoreObs.wireRoute, a padding byte set on every load and ingest path), else the transmission's route unless route_mask is mixed (#308). - heard_by.go: classifier and one-pass accumulator shared by single and bulk health (O(observations), storage reused across bulk nodes). - Handlers drop hidden observer identities with one name lookup. - GetBulkHealth closes its node rows before the store work. - OpenAPI schemas and docs/api-spec.md; named structs replace the untyped observer rows. - public/heard-by.js renders the split for the node page, side panel and Live panel; home My Mesh and Analytics use its counts and texts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
public/heard-by.js exports window.HeardBy, used by nodes.js, home.js, live.js and analytics.js; declare it next to NodeAdverts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Rapport — CS-pve-agent1 PR#451 #76 — head 4fea395Status: Draft, ready for review. All 18 CI jobs on head Evidence tags:
Commits
Requirements
Always
Local runs on this code
CI per jobThe final run is 38570870612 on head
The first run (38567187258, head
The other failures in that run (Verify E2E Shards, Build & Publish) were downstream gates of those two. Remaining
|
Review — CS-pve-agent3 PR#451 — head 4fea395Dom: APPROVE med nits Condition: before merge, rebase to resolve the F0 shard-manifest conflict, and get a green CI run. Independent, read-only review. Evidence tags: [T] = I ran it (test, command, mutant or probe), [K] = I read it in the code or the firmware source, [A] = analysis that I did not execute. Scope of what I checked: head Master moved to Findings
No code-level blocking findings. F0 only needs a rebase plus a green CI run before merge. The points, one by one (issue #76 decision options, the owner's follow-up comment, and the brief)
Always-rules
Tests I ranOn the merged tree
CI per job (run 38570870612, on the merge ref against Mutants (my own; each applied alone to the merged tree, then reverted)
Left for the owner
Not verified
|
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…Observed relay tie-break (#76) stats.avgSnr was computed before the hidden-observer filter, so a node whose direct observers were all hidden reported directObservers [] next to a non-null mean. applyVisibleHeardBy recomputes the mean over the rows that stay, for both /health and /bulk-health. The Analytics E2E now asserts that ties on direct observers are ranked by relay observers. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Rapport — CS-pve-agent1 PR#451 master-sync — head 107f96aReview feedback addressed (commit Evidence tags: [T] = I ran it, [K] = I read it in the code, [A] = analysis, not executed. Findings
Tests
Mutants (each applied alone, then reverted; all killed)
CI (run 38601747188, head
|
Relates to #76
Problem
GET /api/nodes/{pubkey}/healthandGET /api/nodes/bulk-healthreturned oneobserverslist: every observer of any transmission the node is involved in, with the SNR/RSSI of the representative observation. That signal belongs to whichever node the observer last heard, often a repeater, not the node the card is about. The same list fed the node page, the side panel, the home My Mesh card, Analytics "Most Observed Nodes" and the Live node panel.Plan (milestones, all in this PR)
StoreObs.wireRoutekeeps the route type from each observation's own frame header (observations.raw_hex, fix(packets): give each observation its own wire bytes in the detail API (port of upstream #2055) #77). It is one byte placed in existing padding, soStoreObsstays 200 bytes. It is set on all five paths that build an observation: Load, loadChunk, chunked load, IngestNewFromDB and IngestNewObservations.cmd/server/heard_by.go), shared by single and bulk health.directObservers/relayObserversreplaceobservers,stats.avgSnrcovers direct receptions only, hidden identities are filtered, and OpenAPI plusdocs/api-spec.mdare updated.public/heard-by.js) renders the split on the node page, side panel and Live panel. Home My Mesh and Analytics use its counts and texts.scripts/e2e-shards.json, and before/after benchmarks.Config and customizer: this adds no configurable values, so the customizer is unchanged.
The rule
An observation shows that its observer received the node's own transmission only for an ADVERT whose signature is not known to be invalid. It is read with the observation's own route type and path:
Source of truth is the firmware (
src/Mesh.cpp):routeRecvPacketappends the forwarder's hash before it retransmits a flood packet, so the last hop is the node that was heard.sendFloodstarts with an empty path.sendZeroHopsends DIRECT / TRANSPORT_DIRECT withpath_len = 0.removeSelfFromPathbefore retransmitting, so a non-empty direct path is the remaining route.The last-hop credit uses the same identity-safe evidence as the #64 liveness (
relayKeyMatcher), so #64's health semantics are unchanged.The route type is the frame header when one is stored. Without a frame it is the transmission's route type, or unknown when
route_maskholds both flood and direct bits. This is the same rule as the ingestor'sobservationRouteType(#308). The upstream design was read for reference only. It classifies by the representative transmission's route type and counts relay observers from the representative observation only; neither is done here.An observer never counts as hearing its own upload of the node's packet. Every other observer of the node's transmissions goes to
relayObservers, counted from all observations. ItspacketCountis the number of the node's transmissions it saw, and it carries no SNR/RSSI.API
directObservers: HeardDirectObserver[]:observer_id,observer_name,packetCount(direct receptions),avgSnr,avgRssi(over those receptions), andcan_relay(node health only, omitted when unknown).relayObservers: HeardRelayObserver[]:observer_id,observer_name,packetCount,can_relay.stats.avgSnr: the mean over direct receptions; null without one.observersis removed. Keeping the name with a new meaning would silently change every client, and keeping the old content would keep the misleading SNR.map[string]interface{}count:store.go213 → 209, nothing added elsewhere. The OpenAPI schemas use the typedopenAPISchema, and a test pins them to the struct fields.cmd/serverstays read-only.GetBulkHealthalso releases its node rows before the store work. Its earlybreakused to keep that connection whilerelayPrefixMapLockedqueried again, which hangs a single-connection pool. A regression test covers it.Frontend
o.iata, which these endpoints never emitted.The tooltip and docs say that the lists are observed routing evidence, not a radio range.
Performance
The new pass is O(observations of the node's transmissions), with one map lookup per observation. There is no per-node scan of the store, no extra lock and no background job. Bulk health reuses one accumulator across nodes.
BenchmarkHeardBy*uses 2,000 nodes and 30,000 transmissions × 5 observations. Medians of 3 interleaved runs of 20 iterations each, 4-core host. "Before" is master plus only therows.Close()line, without which the benchmark deadlocks.Bulk health takes 2× the time because it now reads every observation instead of one per transmission. That is required to count relay observers from all observations and to read route evidence per observation. The cost is about 34 ns per observation, with less garbage than before.
Tests
cmd/server/heard_by_test.go:wireRouteon all five observation paths;StoreObspadding;repeat: on|offhint to exclude listening-only observers from path-hop candidates Kpa-clawbot/CoreScope#1290 badge test seeds a real direct reception. The golden shapes are updated.test-heard-by.js(intest-all.sh) covers the renderers, tooltips, the empty state and escaping.test-issue-76-heard-by-e2e.js(shard 3) runs against the Go server with the CI fixture, without stubs. It checks:Run locally on this head:
go test -race ./...(server) andgo test ./...(ingestor) pass.sh test-all.sh: 249/249.node test-frontend-helpers.js: 713/713.bash scripts/check-xss-sinks.sh --diff origin/masteris clean.eslint@8 public/*.js(the CI config, with the newHeardByglobal) reports 0 errors and no new warnings in the touched files.e2e-fixture.db:test-e2e-playwright.js: 154/157, 3 skipped;test-a11y-axe-1668.js: 0 violations;Not in this PR
/api/nodes/{pubkey}/analyticsstill has its own observer coverage with representative SNR. It is a different card and is not part of this change.🤖 Generated with Claude Code