perf(store): evaluate transmission-ID keys for byNode deduplication #534

Problem and priority

Measurement-first memory improvement, medium priority. Fork master 8798f74eeec124f21d0e82a8d64b93b31d7facc0 (24 October 2026) still keys the per-node byNode de-duplication sets with transmission hash strings. Evaluate numeric transmission IDs instead, without changing membership or merge behavior.

Proposed adaptation

• Use explicit typed sets/helpers and update every constructor, merge, eviction, load and health-region consumer consistently.
• Prove the one-live-transmission-per-hash assumption for this fork. Handle hash migration, surviving IDs, later observations and chunk merges without losing membership or leaking a stale ID.
• Preserve the recent observation de-duplication work in perf(store): avoid eager per-transmission observation dedup maps #527 / perf(store): avoid eager per-transmission observation dedup maps #411 and accounting fixes. Coordinate with perf(store): evaluate on-demand subpath counts instead of permanent indexes #413 / open perf(store): on-demand subpath counts instead of permanent indexes #529, perf(store): evaluate bounded sharing of observer metadata strings #489 and related store-memory work; do not independently rewrite their branches.
• No schema, API, configuration or workflow changes.

Acceptance criteria

Tests cover duplicate/later observations, two hashes, ID survival/transfer, eviction/re-addition, background chunks, regional health filtering and concurrent readers. Run full server and race tests. Measure representative fork heap, retained set memory, bytes accounted, allocations and CPU before/after; do not quote upstream's savings as ours. Include a regression bound or repeatable benchmark and independent review.

Source provenance

Source repository: Kpa-clawbot/CoreScope
Pull request number: 2199
Head commit: 41f6912b8ecf0d138e2d1a02a0a6216db179d841
Merge commit: cf342efc4336cae56cf8857ecc3dfa9418a63937
Relevant files: cmd/server/store.go and byNode/health consumers