Add benefit-per-byte KV cache admission and eviction policy

[i386]

Context

PR #1632 currently provides reference-aware LRU and hard-budget enforcement. LRU alone does not distinguish an expensive repeatedly reused prefix from a large one-shot prefix that should never have consumed SSD bandwidth.

Mooncake reports highly skewed production reuse, with many blocks never reused and hot blocks accessed repeatedly. Preble co-optimizes prefix reuse and serving load rather than using locality alone.

Coordinated in Buzz channel #skippy-kv and tracked as an expanded optimization candidate for #1632.

Proposed direction

Use measured benefit per exclusive physical byte:

reuse_probability * max(cold_prefill_cost - restore_cost, 0)
-----------------------------------------------------------
                 exclusive_physical_bytes

Credit shared segments correctly, place new/large entries in probation, and persist only after a second-hit or equivalent value signal. Tier routing should compare measured queue + restore + suffix-prefill cost with cold prefill rather than treating every hit as valuable.

Checklist

• Define entry and shared-segment accounting without double-counting physical bytes.
• Add bounded reuse/value statistics and decay so stale historical popularity does not pin data forever.
• Implement probation and second-hit/value-triggered persistence.
• Define admission, promotion, demotion, and eviction scores with deterministic tie-breaking.
• Preserve hard budgets, minimum-free reserve, active pins/holds, exact identity, and cold fallback.
• Expose policy decisions and reasons without logging prompt content or stable content fingerprints.
• Replay realistic turn-growth, one-shot, Zipf/hotset, mixed-size, and concurrency traces.
• Compare LRU and candidate policies on hit value, bytes written, cache-hit TTFT, goodput, and tail latency.

Acceptance gate

Adopt a policy only when the same-capacity matched replay beats current LRU on saved prefill time or goodput, reduces or justifies SSD writes, remains stable under adversarial churn, preserves exact output, and does not regress the #1632 latency gates.