Add configurable node-local disk prompt cache

[i386]

Summary

Nodes can now configure and operate a bounded, durable disk prompt cache without relying on experimental ambient SKIPPY_L3_* settings.

• adds [runtime.kv_cache.disk] with off/auto/fixed modes, absolute node-local roots, hard budgets, and minimum-free reserves
• resolves each field independently through defaults < TOML < public environment < CLI and reports the winning source
• retains one-release legacy environment compatibility without allowing zero/unbounded budgets
• injects one node-scoped manager into every solo and local-split Skippy stage
• applies budget/reserve changes live with rollback on persistence failure; mode/path changes remain restart-required
• adds versioned loopback status, exact numerical-model prune/clear operations, and mesh-llm kv-cache status|prune|clear

This is stacked on #1632. The browser settings UI is intentionally deferred. Authenticated owner-control forwarding and the captured-workload release report remain follow-up commits before this draft is ready.

CLI

mesh-llm serve --kv-cache-disk auto --kv-cache-disk-dir /fast/mesh-kv
mesh-llm kv-cache status --json
mesh-llm kv-cache prune --target 16GiB --yes
mesh-llm kv-cache clear --model-identity blake3:<exact-id> --yes

Architecture

The disk tier remains the concrete node-local writable store. A single process manager owns the canonical root, root lock, reservations, pins, lifecycle gate, accounting, and status. Lifecycle mutation drains existing cache operations; request-side fills decline immediately during that drain so inference continues with cold prefill.

Automatic budgeting uses filesystem_available + current_managed_usage as the stable 20% capacity basis, is capped at 64 GiB, and is constrained by the minimum-free reserve. It can shrink live but does not grow again until restart.

Compatibility

The default remains off. SKIPPY_L3_DIR / SKIPPY_L3_BUDGET_BYTES are field-level fallbacks only when public settings do not supersede them. SKIPPY_L3_BUDGET_BYTES=0 warns and resolves to the bounded 32 GiB compatibility default; it never means unbounded.

Validation

Validated at efb6a26dc9ee6ea37d43dde45b10bd5bd3eb0853:

• formatting and warning-denying Clippy across the touched shipped/cache/server packages
• skippy-cache: 114 passed, 0 failed, 1 ignored measurement
• skippy-server: 676 passed, 0 failed, 3 ignored model-gated tests
• mesh-llm-host-runtime: 2,948 passed, 0 failed, 9 ignored
• mesh-llm-cli: 89 passed, 0 failed
• mesh-llm-config: 193 passed, 0 failed

Still required before ready

• authenticated owner-control status/prune/clear with per-node results
• exact local API/CLI black-box fixtures
• captured Buzz/OpenCode/Goose restart lifecycle and two-machine full-chain warm evidence
• c64/c128/c256, write-amplification, TTFT, and saturation measurements

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

This pull request is currently a draft. Reviews will not take place until the PR is marked as ready for review.

[i386]

Folded directly into #1632 at 2b22c5c per maintainer direction; continuing the remaining non-UI #1576 work there.