Track safe historical key backfill and evidence-gated catalogue keys #192

Context

Upstream proposal: Kpa-clawbot/CoreScope issue 2107.

Our fork can retain enc_* GRP_TXT rows and unnamed transport scopes after an operator adds a channel or region key. New packets can use the new key, but historical transmissions are not automatically reprocessed. The upstream proposal also suggests promoting keys from a public catalogue after local traffic provides evidence.

This is a tracking/design issue for our fork, not authorization to import a catalogue or rewrite production data. Our current fork has the display-only knownChannelsUrl integration but does not yet have upstream's autoRegionKeys; do not assume the upstream implementation can be copied unchanged.

Recommended sequence

1. First PR: explicit-key historical backfill. When operator-configured hashChannels or hashRegions grows, reprocess eligible historical rows in small, resumable, ingestor-owned batches. Preserve raw packets and existing explicit-key precedence. Do not overwrite a resolved row merely because another key collides.
2. Separate design and PR: catalogue key tier, opt-in only. Define catalogue provenance, size/key caps, secret-key policy, removal/rollback semantics, and an evidence gate before any automatic promotion. Keep catalogue-confirmed keys visibly distinct from operator-configured keys. Consider upstream autoRegionKeys separately, since it is not in this fork.

Correctness and safety requirements

• Count distinct transmissions/packet contents, not observation rows, as evidence. Several observers receiving one packet must not make one match look like several independent hits. Test repeated identical packets and 16-bit MAC/transport-code collisions.
• For region backfill, reuse the existing per-packet matching and ambiguity policy: if multiple non-explicit keys match and there is no justified winner, leave the scope unnamed. A catalogue hit is evidence, not operator authorization.
• For channel backfill, verify MAC and decoded payload plausibility, including short/adversarial payloads. Define the threshold for promotion and test false-positive candidates before enabling this tier.
• Reuse the ingestor's decode/serialization path so rewritten decoded_json has the same representation as a newly ingested packet.
• Specify how old-row updates invalidate or refresh the server's in-memory packet store, channel/analytics caches, WebSocket-visible data, and derived node fields such as default_scope. Polling only new transmission IDs is insufficient.
• Bound database work and writer-lock hold time; checkpoint progress so a crash/restart resumes safely. Expose progress, errors, and counts. Test against a staging-size copy before any deployment.
• Keep the feature off by default. No staging or production backfill without a separate review and rollout/rollback plan.

Done when

• The explicit-key backfill has failing-first tests for legacy rows, duplicate observations, collision/ambiguity, idempotence, crash-and-resume, and dependent-state refresh.
• Benchmarks on a realistic DB copy show bounded batches without unacceptable live-ingest delay.
• Catalogue promotion, if pursued, has its own reviewed trust model and validation PR. Do not merge it into the first backfill PR.

No implementation or deployment is requested by this issue alone.