Add cross-request LRU prompt cache (PromptTrie + byte-bounded eviction)

[GoodOlClint]

Proposed changes

Swift only reuses a KV cache within a single ChatSession. A new request whose prompt shares a prefix with an earlier one re-processes that prefix from scratch. Python mlx_lm has had a cross-request store for exactly this — PromptTrie + LRUPromptCache in mlx_lm/models/cache.py, used by server.py — and this ports it to Swift.

What's added

• KVCache.nbytes (Libraries/MLXLMCommon/KVCache.swift) — byte accounting, the prerequisite for any byte-bounded policy. Every Python cache implements nbytes; no Swift cache did. The basis is the allocated (step-rounded) buffers from innerState(), not the offset-sliced state, matching Python's KVCache.nbytes.

  It is declared as an open var on BaseKVCache rather than left to the protocol-extension default, so subclasses get a dynamically-dispatched override point. (A protocol-extension default becomes the witness at BaseKVCache's conformance and a subclass property would only shadow it statically.)

• PromptTrie — prefix trie returning the nearest stored sequence: exact / shorter / longer, plus the common-prefix length.

• LRUPromptCache — the store itself: fetchNearestCache (returns a deep copy plus the token remainder still to process), prefix trim-reuse, maxSize / maxBytes eviction, type-aware eviction ordering (assistant → user → system, so system prompts survive longest), and statsByType().

Two Python bugs fixed at port time

The port deliberately does not inherit two bugs still open in the Python implementation — reported in ml-explore/mlx-lm#1495 and fixed there by ml-explore/mlx-lm#1496:

1. A value stored at a single-token prefix never matched (search used > 0 where >= 0 is correct).
2. Eviction ignored fetch recency, making the store FIFO rather than truly LRU.

Both are pinned by regression tests (PromptTrieTests, LRUPromptCacheTests).

Scope / notes

• No existing behavior changes. This adds new types plus one protocol requirement that ships with a default implementation. Nothing is wired into ChatSession / Evaluate — that is deliberate, to keep this reviewable on its own and to stay out of the way of Reuse cached chat transcript prefixes #370, which approaches prefix reuse at the ChatSession level. The two are complementary rather than competing: Reuse cached chat transcript prefixes #370 adds supportsStaticPrefixReuse and this adds nbytes; both touch only the KVCache protocol block, so they should merge cleanly in either order.
• LRUPromptCache is a non-Sendable final class, mirroring Python's single-server-thread assumption. Cross-task consumers wrap it in SerialAccessContainer, as ChatSession already does elsewhere.

Checklist

Put an x in the boxes that apply.

• I have read the CONTRIBUTING document
• I have run pre-commit run --all-files to format my code / installed pre-commit prior to committing changes
• I have added tests that prove my fix is effective or that my feature works
• I have updated the necessary documentation (if needed)

🤖 Generated with Claude Code