You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
fix(store): make trackedBytes accurately enforce maxMemoryMB #113
Evaluate and adapt Kpa-clawbot/CoreScope#2035: correct the in-memory store accounting used by packetStore.maxMemoryMB.
Current fork evidence
Verified against fork master 1bdf088e:
StoreTx has no persistent record of how many bytes it was charged.
A transmission is charged before pickBestObservation assigns PathJSON, so path-dependent byPathHop and O(path²) subpath costs are omitted.
Eviction later recomputes the estimate after the path exists and can subtract more than was originally added, drifting trackedBytes downward.
The estimate omits the decoded cache, separately allocated dedup-key strings, and relevant observation strings.
docs/go-migration.md still incorrectly says the Go server ignores packetStore.maxMemoryMB.
An underestimated counter can prevent intended eviction and leave the real heap pressed against its runtime/container limit.
Fork-specific adaptation
Do not cherry-pick blindly. The fork pins unsafe.Sizeof(StoreTx{}) to 320 bytes for route-mask padding and has additional indexes/caches. Preserve that layout with a justified design or update the contract deliberately, and recalibrate constants on this branch.
Acceptance criteria
Record, directly or equivalently, the exact estimate charged per transmission.
Apply only deltas when selected paths or other charged allocations change.
Eviction subtracts exactly the amount charged.
Include path/subpath indexes, decoded-cache allowance, dedup-key allocations, and relevant observation strings.
Deterministic insert → path assignment → recharge → eviction returns the counter to exactly zero.
Mixed insert/update/eviction tests show no drift; memory eviction reliably crosses high and reaches low watermarks.
Cold-load budgeting and documentation use the revised estimate.
Existing route-mask, index, and cache behavior remains intact.
Verification
Run focused and full server tests under -race, static checks, before/after tracked-vs-heap measurements on a read-only production-sized copy, cold-load/eviction measurements at representative limits, and insert/eviction performance checks.
No deploy or live-environment mutation is part of this issue.
Fixed by #198 (merged as 81039e35). trackedBytes now records the exact charge per transmission and charges only deltas when the selected path changes. Eviction subtracts exactly what was charged, and the estimate includes paths, subpaths, the decode cache, dedup keys and observation strings. The tracked/heap ratio went from about 0.5–0.6 to about 1.0 on synthetic 100K and 500K stores.
Leftovers are tracked in #202: a hash-migration double credit and the uncharged path_json fallback, both pre-existing. Confirming the ratio on real data is part of the next staging round.
Activity
test(store): cover the production startup path and pin the new estima…
Merge pull request #198 from dborup/codex/issue-164-113-tracked-bytes
dborup commentedon Oct 4, 2026
Fixed by #198 (merged as
81039e35).trackedBytesnow records the exact charge per transmission and charges only deltas when the selected path changes. Eviction subtracts exactly what was charged, and the estimate includes paths, subpaths, the decode cache, dedup keys and observation strings. The tracked/heap ratio went from about 0.5–0.6 to about 1.0 on synthetic 100K and 500K stores.Leftovers are tracked in #202: a hash-migration double credit and the uncharged
path_jsonfallback, both pre-existing. Confirming the ratio on real data is part of the next staging round.