perf(store): evaluate in-place resolved-path pubkey scanning and bounded sharing #533

Problem and priority

Measurement-first memory/CPU improvement, medium priority. Fork master 8798f74eeec124f21d0e82a8d64b93b31d7facc0 (24 October 2026) still has bulk-load/eviction paths that decode resolved-path JSON to collect non-empty pubkeys. Evaluate scanning the usual representation in place, with a correctness-preserving JSON fallback.

Fork adaptation and dependencies

• Inventory current loaders, eviction and existing decodePersistedRelayPath optimizations before changing anything. Preserve public-key validation, canonicalization, hash-width handling, ordering and duplicate semantics.
• Coordinate with perf(store): evaluate bounded sharing of observer metadata strings #489 (bounded string sharing) rather than adding a second pool. The source implementation depends on interning that our fork does not yet have.
• Any retained new string must not pin a large input buffer. Saturation and unusual/escaped JSON must fall back safely. Do not introduce an unbounded cache or hot-lock contention.
• Keep trackedBytes, eviction and ownership meaningful; distinguish logical charges from retained bytes and the pool's own retention.

Acceptance criteria

Compare scanner vs current decoder on malformed/escaped JSON, nulls, empty keys, mixed values and randomized inputs; run race and all affected load/merge/eviction tests. Benchmark current fork before/after with realistic observation counts and report allocation count, allocated bytes, retained heap, peak RSS and startup/eviction CPU separately. Source measurements are not measurements of our instance. No schema/API/config/deployment change should be bundled.

Source provenance

Source repository: Kpa-clawbot/CoreScope
Pull request number: 2197
Head commit: a7f00ad65d680e3d8c539dba797955b9e222d6ec
Merge commit: 9529bf9d4188ab588488aa4fd81b37398e953cda
Relevant files: cmd/server/store.go, cmd/server/chunked_load.go