lru: accounting per-key overhead

[AskAlexSharov]

lru accounting logic has bug: didn't account overhead metada per-key. but in some cases our keys/vals are small and it's noticable

cache	budget	before	after
accounts	150 MB	1,638,400	418,816
storage	1 GB	12,201,611	3,354,624
jumpdest	64 MB	262,144	104,448

At 256 shards. The ceiling now moves with the shard count, because each shard's
table is fitted to the 5/4 boundary rather than charged a flat per-slot constant —
that flat charge was wrong by up to ±292 MB on the storage cache purely as a
function of GOMAXPROCS.

Grow steps are charged from the real table on both sides of the step, not from the
fitted ceiling: at a power-of-two generation freelru allocates 2x the capacity, so
an intermediate generation costs about double what it used to be charged. That is
what it was really allocating, and it means a cache can now be refused a step it
used to get.

[domiwei]

Reviewed the accounting change — the direction is right, and I verified the table in the description: all three before/after numbers recompute exactly from the new formula. The new test really pins the bug (pre-fix it reserves 262144×88 ≈ 23 MB against ~61 MB actually allocated, so it fails red). Envelope symmetry also checks out: birth / fundGrow / refundGrow / Clear / Close / Purge all go through the single perSlot field, so the reservation always equals curCap×perSlot.

Three things worth addressing (details in the inline comments):

1. The envelope still under-reserves the slot array in reachable default configs — capFitsTable is a near-identity in practice, and freelru's per-shard ceil split can double every shard table (storage default at 256 shards: real table 928 MB vs 742 MB charged).
2. The new maxCacheBytes clamp silently caps user-configured budgets (STATE_CACHE_*, StateCacheBudget) above 1 GiB — an existing 4 GB override now behaves exactly like 1 GB, with no log.
3. The code-size layer's entry ceiling silently drops from 1,000,000 to 216,216, because its budget is synthesized as entries*codeSizeEntryBytes expecting the old divide.

Smaller nits:

• freelruSlotBytes' 112 is a hardcoded mirror of freelru's element size — exact for entry[[]byte] and codeEntry, conservative for codeSizeEntry (96 B). An unsafe.Sizeof static assert would keep a future larger value type from under-charging silently.
• "Stacked on #23546" in the description is stale — that PR was closed unmerged, and this diff is clean against main.

[yperbasis]

@AskAlexSharov It says "Stacked on #23546", but 23546 is closed?

[AskAlexSharov]

@AskAlexSharov It says "Stacked on #23546", but 23546 is closed?

stale description

[AskAlexSharov]

The two smaller nits are closed; head is 1a2a53e981. The three numbered issues were fixed earlier in the inline threads.

• freelruSlotBytes' hardcoded 112 — freelru.element is unexported, so the assert goes on the value types that determine its size:

  const freelruValueBytes = 72
  const (
      _ = uint(freelruValueBytes - unsafe.Sizeof(entry[[]byte]{}))
      _ = uint(freelruValueBytes - unsafe.Sizeof(codeEntry{}))
      _ = uint(freelruValueBytes - unsafe.Sizeof(codeSizeEntry{}))
  )

  Checked it bites: dropping the constant to 71 fails the build on entry[[]byte] and codeEntry (both 72 B) and passes codeSizeEntry (56 B, the conservative case you named). A larger value type can no longer under-charge silently.

• "Stacked on lru: fastpath before taking resizeMu in maybeGrow #23546" — gone from the description.

[Copilot]

Pull request overview

Updates LRU memory accounting to include freelru metadata overhead.

Changes:

• Adds per-slot overhead calculations and capacity fitting.
• Preserves entry-count semantics for the code-size cache.
• Adds allocation-envelope coverage testing.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File	Description
execution/cache/grow_lru.go	Adds overhead-aware grow-LRU sizing.
execution/cache/generic_cache.go	Accounts for freelru slot metadata.
execution/cache/generic_cache_concurrency_test.go	Tests envelope allocation coverage.
execution/cache/code_cache.go	Uses an entry-count constructor for code sizes.

Suppressed comments (2)

execution/cache/grow_lru.go:80

• The byte-derived slot count is narrowed to uint32 before it is capped. Once the quotient exceeds math.MaxUint32, it wraps and may create a much smaller cache rather than selecting maxCacheSlots. Apply the cap while the value is still uint64.

	perSlot := int64(avgBytes) + freelruSlotBytes
	maxCap := max(fitTableSlots(min(uint32(uint64(maxBytes)/uint64(perSlot)), maxCacheSlots)), 1)
	return newGrowLRUWith(maxCap, perSlot, onEvict)

execution/cache/generic_cache.go:94

• The quotient is converted to uint32 before applying maxCacheSlots. For a valid large byte budget where the quotient exceeds math.MaxUint32, this wraps and can produce a tiny cache instead of saturating at the configured ceiling. Clamp in uint64 first, then convert.

	perSlot := uint64(payloadBytes) + freelruSlotBytes
	approx := min(uint32(uint64(capacityBytes)/perSlot), maxCacheSlots)
	shards = initialShardCount(approx, shardCeil())

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[yperbasis]

Requesting changes for two accounting regressions. Intermediate grow generations still under-reserve the freelru table, so the shared memory envelope can be exceeded. Also, the byte-usage metric no longer shows when the cache reaches its new overhead-aware entry ceiling. I reproduced the first issue on this head; the existing execution/cache tests pass because the new envelope test measures only the fitted final generation.

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.

Suppressed comments (1)

execution/cache/generic_cache.go:251

• avgBytes still includes the 24-byte per-entry overhead (putStriped and the account/storage averages all add it), while freelruElemBytes now charges for the complete freelru element, including the inline entry[T]. Treating the unchanged estimate as external payload therefore double-counts that overhead, reducing the account/storage ceilings and consuming the shared envelope early. Split the estimates into externally allocated key/value bytes versus table-resident bytes, or adjust the callers before using them as payloadBytes.

		payloadBytes: int64(avgBytes),