feat(skippy-cache): bounded host-RAM L2 tier prototype (#1651) by i386 · Pull Request #1749 · Mesh-LLM/mesh-llm

i386 · 2026-09-10T04:24:45Z

First slice of #1651, stacked on #1736 (7247a397a). This establishes the bounded host-RAM L2 ownership and integrity contract without touching the request restore path.

What

skippy-cache::l2 — a bounded host-RAM tier over immutable, content-addressed segment handles:
- Entries reuse L3 namespace and token-path coordinates plus model and exact-state identities.
- Prefix growth shares segment handles, so the byte budget charges distinct physical segment bytes once while stats expose both physical and logical footprint.
- Admission validates the whole-payload digest, each segment digest, manifest tiling, indices, offsets, payload geometry, and existing segment identity before mutating the pool.
- Admission and same-key replacement are journaled transactions. Incoming handles remain protected through replacement and eviction; an admission that cannot fit rolls back completely with ProtectedOvercommit.
- Deterministic ownership accounting assigns every pooled segment to exactly one live entry and recomputes after admission, eviction, removal, corruption cleanup, and rollback. Eviction reports physical bytes freed separately from shared bytes retained.
- peek is side-effect free. A successful get advances recency; absent or corrupt entries become cold misses without heating the LRU.
- Budget enforcement is deterministic LRU with stable key tie-breaking. Empty, oversized, malformed, corrupt, and unknown-kind payloads are refused without leaving partial state.
skippy-bench l2-tier — compares cold L3 fill with warm L2 lookup on the identical packed entry. Both arms start before lookup and end at usable materialized bytes, with byte equality required before timing is reported.
- User-supplied store roots are refused if they already exist. Benchmark cleanup only removes roots it created and sentinel-marked.

Request-path promotion, demotion, server integration, and discrete-CUDA qualification remain later #1651 slices.

Bench numbers (M1 Ultra, release build, 50 pairs, head `f678ce337`)

Both arms use the same nothing-to-usable-bytes boundary. L3 includes index lookup, segment reads, assembly, and digest verification. l2_lookup_to_usable_bytes_ns includes L2Tier::get, handle assembly, and materialization; handle-only lookup is reported separately. These figures supersede all earlier L2 numbers in this PR and its comments.

workload	L3 fill p50	L2 lookup-to-usable p50	speedup
1,893 tokens × 512 B/token (~0.9 MiB)	1.412 ms	26.083 µs	54.13x
3,994 tokens × 4 KiB/token (~15.6 MiB)	20.258 ms	536.250 µs	37.78x

Both runs produced 50/50 L2 hits with zero misses or evictions. This is a unified-memory microbenchmark; request-path TTFT/goodput and discrete-CUDA transfer behavior are not measured here.

Validation

Validated in a clean worktree with local branch, remote branch, and GitHub PR head all at f678ce337236e18613d434713db107465bcc65ef:

cargo test -p skippy-cache: 150 passed, 1 ignored, 0 failed; doc tests passed
cargo test -p skippy-bench: 85 passed, 0 failed
cargo clippy -p skippy-cache -p skippy-bench --all-targets -- -D warnings: clean
cargo fmt --all -- --check: clean
git diff --check origin/scama/skippy-l3-streaming-restore...HEAD: clean
console-print consistency check: clean
L2 benchmark smoke: 2/2 hits, byte equality preserved, owned temporary store removed

…gate The 128 default (PR #564, no recorded rationale) diverged from llama.cpp's own LLAMA_SERVER_DEFAULT_N_UBATCH = 512 and missed the CUDA SSM SSD kernel gate (n_tok > SSM_SSD_MIN_TOKENS, 128, strict) by exactly one token on every default recurrent prefill, forcing the sequential-scan fallback. Measured on granite-4.0-h-1b (2026-09-08 competitive bench, same binary and protocol): TTFT p50 0.670 -> 0.415 s (C1) and 6.38 -> 3.97 s (C8), C8 decode 22.2 -> 39.4 tok/s. Dense negative control (Qwen3-1.7B) flat. Cost: +203 MiB CUDA compute buffer. Also aligns the gpu-tune planner copy, corrects the setting description (physical prefill chunk size, not decode micro-batch), and forwards the resolved n_ubatch / flash_attn llama_context lines into mesh.log so config landing is observable without buffer-size fingerprinting.

Frozen-conversation benchmark that measures serving latency across a full process restart: fill (cold server, growing multi-turn prefix), restore (SIGINT, fresh serve on the same state directory), and warm (repeat replay without restart). Server starts with production defaults; the only extra arguments are an explicit --serve-extra-args pass-through so a durable KV tier can be A/B-measured without touching the harness. Per-run provenance (source SHA, binary/model SHA-256, hardware fingerprint, manifest SHA-256) plus JSONL request rows and a Markdown report land in the output directory. Verified end to end on darwin/aarch64 (Apple M2, SmolLM2-135M-Instruct Q8_0): fill cache 61%, restore cohort captured across a measured 7s restart, warm cache 100%, zero failed requests.

- fill prefixes now end on the user turn being answered (CodeRabbit #454) - restore cohort records only the first post-restart replay; subsequent replays are resident-warm and recorded under the warm cohort (#499) - forbidden-startup-options check also rejects --opt=value forms (#166) - stream failures degrade to per-request errors instead of aborting (#348) - missing git degrades provenance instead of aborting (#413)

…1741)

…ng (#1744)

github-actions · 2026-09-10T04:24:52Z

This pull request is currently a draft. Reviews will not take place until the PR is marked as ready for review.

coderabbitai · 2026-09-10T04:25:01Z

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: eabb6fb1-5475-4351-b9fd-78f66020a6ee

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

First slice of #1651: a standalone, opt-in L2 tier over the packed L3 segment format, with no restore-path edits so it cannot collide with #1649. - crates/skippy-cache/src/l2: L2Tier stores assembled exact-state payloads under the same (namespace, token-path) coordinates and model/state identities L3 uses. Entries keep the whole-payload BLAKE3 digest and reads verify it; a mismatch drops the entry and records a miss, never serves state. The byte budget is enforced on every insert with deterministic LRU eviction (ties break on cache key); oversized and empty payloads are refused, mirroring L3 spill rules. Reads clone Arc-backed CacheBytes handles rather than bytes. - crates/skippy-bench l2-tier: cold L3 fill versus warm L2 fill on the same packed entry, with a byte-equality correctness gate before any timing is reported. On this machine, 3,994-token x 4 KiB/token entries: L3 fill p50 22.3 ms versus L2 fill p50 1.75 us. Validation: cargo test -p skippy-cache --lib (132 passed), clippy -D warnings, fmt --check.

* label the tile Mesh Capacity and drop the rated meta line * exclude client-role nodes from Mesh Capacity totals * resolve client peers the same way for rows and totals * note why isClientPeer stays separate from resolvePeerRole

Review corrections on the first #1651 slice (PR #1749): - Store immutable Arc segment handles keyed by content digest plus a prefix-to-segment layout, so a longer prefix shares the prefix bytes of the shorter entry instead of duplicating them. Turn growth no longer trends toward quadratic RAM; the budget is charged with distinct segment bytes only. - Make the integrity claim true: admit() verifies the concatenated L3 wire digest (segment_digest) exactly once at admission and refuses mismatched bytes (DigestMismatch), so Direct origin can no longer admit arbitrary bytes under a valid-looking digest. Timed reads are digest-keyed verified-handle lookups, not digest checks. - peek() is now side-effect free: it never touches LRU recency. Recency moves only on a successful get(). - Bench harness rejects zero pairs/tokens/bytes-per-token and uses checked arithmetic for payload length and MiB conversion; the byte-equality gate stays outside the timer; the L2 metric is renamed to lookup/assembly-handle time with one-time admission hashing reported separately.

…ed (#1651) Review corrections round 2 on PR #1749, all against 247a4a5: - admit() is transactional over the segment pool: the incoming layout's handles are installed or pinned (protected set) before the replaced entry is released and before eviction runs. An identical-wire same-key re-admit can no longer delete its own shared handles, and an admission that shares its eviction victim's segments can no longer have those handles dropped mid-transaction. Retained shared bytes transfer to the new entry's budget charge. - Pool reuse trusts content, not digest text: every layout segment slice is BLAKE3-hashed against its exact wire range before any pool mutation (validate_layout), the layout must tile contiguously with kv+recurrent == total, and from_manifest now checks segment index and offset. Same-digest/different-bytes offers are refused (SegmentDigestMismatch / ConflictingSegment) unless they replace the same key; a referenced content-addressed handle is never replaced. - get() with a missing segment handle is a cold miss: the corrupt entry and its surviving handles are removed, LRU never moves, and the miss counter (previously never incremented) now moves on every miss, including absent keys. - Bench: both arms are timed through the same usable-bytes boundary (handle lookup reported separately as l2_handle_lookup_ns); the store root is bench-owned — pre-existing paths are refused instead of deleted, the created root is sentinel-marked, and cleanup refuses unmarked directories (regression test included). Validation on this head: cargo test -p skippy-cache 144 passed / 1 ignored; cargo test -p skippy-bench 85 passed; cargo clippy -p skippy-cache -p skippy-bench --all-targets -- -D warnings clean; cargo fmt clean.

i386 · 2026-09-10T05:51:46Z

Review round 2 pushed at 608ec6c38 (single commit on 247a4a5db, still stacked on scama/skippy-l3-streaming-restore).

Atomic admission. admit installs or pins every incoming segment handle (a protected set) before the replaced entry is released and before eviction runs. An identical-wire same-key re-admit can no longer delete its own handles (regression: three rounds of re-admit must serve the full wire with segments=4, bytes=64 throughout), and an admission that shares its eviction victim's segments keeps them (regression: budget total-1 forces the victim out; the shared prefix must survive, freed accounting exact). Retained shared bytes transfer to the new entry's budget charge.

Content-verified pool reuse. validate_layout BLAKE3-hashes every segment slice against its exact wire range before any pool mutation; the layout must tile contiguously with kv+recurrent == total; from_manifest now checks index and offset per segment. Same-digest/different-bytes offers are refused (SegmentDigestMismatch / new ConflictingSegment) unless they replace the same key — a referenced content-addressed handle is never swapped underneath a live entry (regression included; the live victim's handle is verified intact).

Corrupt-entry get. A missing handle is a cold miss: entry + surviving handles removed, LRU untouched, misses counter incremented (it was previously never incremented anywhere; absent-key misses count too).

Bench. Both arms timed through the same usable-bytes boundary; l2_lookup_to_usable_bytes_ns is the headline with l2_handle_lookup_ns separate. Store root is bench-owned: pre-existing paths refused (never deleted), sentinel-marked root, cleanup refuses unmarked directories (test covers refusal + survival of user files).

Fresh numbers (M1 Ultra, release, 50 pairs, 1,893 tokens × 512 B): L3 fill p50 1.31 ms vs L2-to-usable-bytes p50 18.5 µs (~71x), handle-only p50 3.0 µs, one-time admission hash 1.2 ms. (Single-segment dedup in the pool: 15 segments / 969,216 distinct bytes.)

Validation at 608ec6c38: cargo test -p skippy-cache 144 passed / 1 ignored / 0 failed (+5 regressions), cargo test -p skippy-bench 85 passed / 0 failed, cargo clippy -p skippy-cache -p skippy-bench --all-targets -- -D warnings clean, cargo fmt clean.

…ined accounting (#1651) Review corrections round 3 on PR #1749, all against 608ec6c: - Hard byte cap: admission is a journaled transaction (AdmitJournal). Reservations, same-digest overwrites, orphan releases, the replaced entry, and evictions are all recorded; if eviction cannot bring the final pool under budget while the admission's own handles are pinned (e.g. an incoming entry sharing bytes with its victim), the journal is rolled back exactly — entries, handles, pool bytes, evictions counter, and charges — and the admission is refused with the new ProtectedOvercommit refusal. The pool can never exceed the budget. - Benchmark boundary: the L2 timer now starts before l2.get, so l2_lookup_to_usable_bytes_ns covers lookup through materialized bytes — the same nothing-to-usable-bytes boundary as the L3 arm. The same get is timed inner as l2_handle_lookup_ns. - Exact charges: entry charge_bytes are recomputed deterministically from the live map after every mutation (recompute_all_charges): each pooled segment is assigned exactly once to its lowest-cache-key live reference. The sum of charges always equals the physical pool after admission, eviction, removal, corruption cleanup, and rollback. - Exact retention: remove() and evict_to_limit() report retained_bytes from actual post-removal pool references (plus admission pins during eviction), never logical-minus-freed. segment_digests() dedupes, so a removed layout referencing X twice with a survivor reports size(X) once. - Ratchet: regen no-console-print allowlist (l2_tier.rs summary print was created after the last regen; main.rs lines had moved). New regressions: admission reusing protected bytes over the budget is refused and leaves the tier untouched; eviction frees only the victim's exclusive segments while pinned shared bytes survive byte-exact; survivor charge covers the pool after the sharer leaves; zero-charge sharer removal never inflates survivor charges (two- and three-sharer cases); repeated same-digest layouts report retained from the pool, with and without a survivor. Validation on this head: cargo test -p skippy-cache 150 passed / 1 ignored; cargo test -p skippy-bench 85 passed; cargo clippy -p skippy-cache -p skippy-bench --all-targets -- -D warnings clean; cargo fmt --check clean; just no-console-print passes.

i386 · 2026-09-10T07:01:31Z

Exact-head benchmark rerun at f678ce337236e18613d434713db107465bcc65ef, with local HEAD and origin/jy/skippy-l2-host-tier verified equal in the same shell.

The corrected L2 timer now begins before L2Tier::get, so l2_lookup_to_usable_bytes_ns includes lookup, handle assembly, and materialization. The L3 arm includes lookup, disk assembly, and verification to the same usable-byte boundary.

M1 Ultra, release build, 50 matched pairs:

Shape	Payload	L3 fill p50	L2 lookup-to-usable p50	L2 handle lookup p50	p50 ratio
1,893 tokens × 512 B	969,216 B	1.412 ms	26.083 µs	3.791 µs	54.13x
3,994 tokens × 4 KiB	16,359,424 B	20.258 ms	536.250 µs	95.833 µs	37.78x

The one-time admission hash was 1.503 ms and 33.236 ms respectively. Both runs had 50/50 L2 hits, zero misses, and zero evictions.

These supersede the earlier figures that excluded lookup. They validate the standalone tier operation on unified-memory Metal hardware. They do not establish the #1651 end-to-end promotion gate: the slice is not wired into the request path, and discrete-CUDA PCIe/import, concurrency, memory pressure, and TTFT/goodput evidence remain later work.

# Conflicts: # crates/mesh-llm-host-runtime/Cargo.toml # crates/skippy-cache/Cargo.toml

Preserve routable aliases for same-model workers, serialize packed GC with publication, reconcile direct store opens, and harden replay and canary validation boundaries.

…e' into jy/skippy-l2-host-tier

Update transitive-memory assertions for omission clearing and compare self-fill output with the public aliases that peers actually advertise.

danielwinterw

Approving on the delta.

The hard part here is admission atomicity and the accounting stays coherent through it. Reserving the incoming handles before releasing the replaced entry, pinning them in protected_set so neither release nor eviction can drop bytes the transaction is about to reference, and journalling every pool mutation so a ProtectedOvercommit rolls back exactly — that is the right shape, and the same-digest-different-content case is handled properly (allowed only when replacing the same key, refused otherwise so a live entry never has a handle swapped underneath it). validate_layout hashing every segment against its exact wire range before touching the pool is the correct ordering for a content-addressed store. peek staying off the LRU clock while get bumps it is right for prefix probing. The corruption path in get — missing handle downgrades to a miss, drops the entry, leaves recency alone — fails in the safe direction.

Note on CI: Rust tests (batch-1) is red with three failures in mesh peer_state and moa_gateway self_fill. Nothing in this diff reaches them; they are the same three failing on #1747, which shares the base, and they originate in #1736.

Nice-to-haves:

recompute_all_charges walks every pooled segment × every entry, and segment_digests() is itself O(D²) because it dedups with Vec::contains. It is called on every admission, every removal, and once per victim inside evict_to_limit, so a single admission that evicts V entries is roughly O(V · S · E · D²). Fine for a prototype driven by the bench harness, but this wants a reverse index (digest → referencing keys) before the tier goes anywhere near the request path — worth a TODO pointing at that so the next slice does not inherit it silently.
L2Hit::wire_view falls back to Arc::new(Vec::new()) when a digest is not in self.segments, then clamps from/to to that zero length. The resulting CacheBytes still reports end - start as its length, so a missing handle would yield a short payload claiming full length rather than an error. get makes this unreachable today, but silent truncation is an unpleasant failure mode for a state-restore path — a debug_assert! or an Option return would be cheap insurance.
On the ProtectedOvercommit rollback path, stats.evictions.fetch_sub(rolled_back_evictions) is not atomic with the fetch_adds inside evict_to_limit, so a concurrent stats() can observe evictions that were rolled back. Only telemetry, but it makes the counter non-monotonic, which is surprising for anything scraping it.

…e' into jy/skippy-l2-host-tier

…e' into fix-i386-1749 # Conflicts: # .agents/skills/manage-ci/references/current-inventory.md # .github/workflows/llama-upstream-canary.yml # Cargo.lock # Cargo.toml # ci/ci.md # ci/llama-canary/agent-repair-prompt.md # crates/mesh-client/Cargo.toml # crates/mesh-llm-api-client/Cargo.toml # crates/mesh-llm-api-server/Cargo.toml # crates/mesh-llm-cli/Cargo.toml # crates/mesh-llm-commands/Cargo.toml # crates/mesh-llm-config/Cargo.toml # crates/mesh-llm-config/src/model/built_in_schema/presentation.rs # crates/mesh-llm-config/src/model/built_in_schema/setting_schema.rs # crates/mesh-llm-console-server/Cargo.toml # crates/mesh-llm-embedded-runtime/Cargo.toml # crates/mesh-llm-hardware-profile/Cargo.toml # crates/mesh-llm-host-runtime/Cargo.toml # crates/mesh-llm-host-runtime/src/mesh/mod.rs # crates/mesh-llm-host-runtime/tests/fixtures/config_schema_reference.json # crates/mesh-llm-log-store/Cargo.toml # crates/mesh-llm-native-runtime/README.md # crates/mesh-llm-node/Cargo.toml # crates/mesh-llm-nodejs/Cargo.toml # crates/mesh-llm-runtime-install/Cargo.toml # crates/mesh-llm-sdk/Cargo.toml # crates/mesh-llm-sdk/README.md # crates/mesh-llm-system/Cargo.toml # crates/mesh-llm-tui/Cargo.toml # crates/mesh-llm-ui/package-lock.json # crates/mesh-llm-ui/package.json # crates/mesh-llm-ui/src/features/network/api/status-adapter.ts # crates/mesh-llm-ui/src/lib/vram.test.ts # crates/mesh-llm-ui/src/lib/vram.ts # crates/mesh-llm/Cargo.toml # crates/mesh-mixture-of-agents/Cargo.toml # crates/mesh-native-serving-plugin-host/Cargo.toml # crates/model-artifact/Cargo.toml # crates/model-hf/Cargo.toml # crates/model-package/Cargo.toml # crates/model-resolver/Cargo.toml # crates/openai-frontend/Cargo.toml # crates/skippy-cache/Cargo.toml # crates/skippy-model/Cargo.toml # crates/skippy-protocol/Cargo.toml # crates/skippy-runtime/Cargo.toml # crates/skippy-scheduler/Cargo.toml # crates/skippy-server/Cargo.toml # docs/SDK.md # docs/design/NATIVE_RUNTIMES.md # docs/plugins/exemplars/web-ui/Cargo.lock # docs/sdk/rust.md # docs/sdk/swift.md # docs/specs/vram-accounting.md # scripts/llama-canary-agent-repair.sh # scripts/tests/test_llama_canary_agent_repair_contract.py # scripts/tests/test_llama_upstream_canary_contract.py # sdk/kotlin/README.md # sdk/kotlin/build.gradle.kts # sdk/kotlin/example/example-jvm/build.gradle.kts # sdk/node/package.json # sdk/swift/README.md # sdk/swift/scripts/generate-swift-bindings.sh # website/src/docs/pages/CLI.md # website/src/docs/pages/developing-plugins.md

i386 · 2026-09-14T22:12:11Z

Superseded by consolidated integration PR #1838. The focused branch and review history remain available; further production wiring continues from the consolidated head.

Paul Hogan and others added 15 commits September 10, 2026 09:46

Spread concurrent new sessions with local route reservations (#1631)

cb871b4

fix(evals): make restart replay cohorts valid

f52ba5d

fix(skippy): preserve resident KV fast path with L3

4e93457

fix(skippy): prefer longer durable restore

e43f42e

fix(skippy): preserve durable restart checkpoints

c9bb7c7

chore(ci): refresh console print ratchet

25cdc0a

perf(skippy): stream packed restore into payload

e03de61

perf(skippy-cache): hash payload during packed restore

cf35b63

Report accurate parameter sizes for local GGUF models (#1742)

114ab68

fix(certification): use served package ref for runtime smoke gates (#…

2080997

…1741)

fix(system): correct day/month transposition in macOS ps lstart parsi…

efd7ab6

…ng (#1744)

perf(skippy-server): prewarm generation graph before readiness

7247a39

i386 added the skippy-kv Work coordinated in Buzz #skippy-kv label Sep 10, 2026

i386 mentioned this pull request Sep 10, 2026

KV roadmap rollup: Skippy KV cache tiers, codecs, reuse, and observability #1732

Open

jy and others added 3 commits September 10, 2026 14:30

i386 force-pushed the jy/skippy-l2-host-tier branch from 1e9a232 to 247a4a5 Compare September 10, 2026 05:17

i386 changed the base branch from scama/skippy-l3-packed-store to scama/skippy-l3-streaming-restore September 10, 2026 05:20

github-actions Bot and others added 2 commits September 10, 2026 06:15

v0.76.0: prepare release source

911d56f

i386 marked this pull request as ready for review September 10, 2026 07:09

ndizazzo and others added 12 commits September 10, 2026 14:36

test(ci): guard product smoke selector drift

6c097d6

ci: preflight packaging dispatch credentials

0520150

ci: publish SDK smoke cache policy

2877ac7

fix: retain device placement in native log summaries

1a2545d

ci: stop duplicate release tag runs (#1731)

0bda645

Merge remote-tracking branch 'origin/main' into sync-tmp-1736

9f4b3ea

# Conflicts: # crates/mesh-llm-host-runtime/Cargo.toml # crates/skippy-cache/Cargo.toml

fix(gpu-tune): preserve throughput profile ubatch

aec8c50

ci: sync protected SDK cache policy

3354456

chore(skippy): restack durable L3 on #1707

7e7a675

chore(skippy): consolidate durable L3 review

6f10192

chore(skippy): validate consolidated durable L3

ab24050

ci: synchronize conventional commit validator

1414471

i386 force-pushed the scama/skippy-l3-streaming-restore branch from f0924b9 to 1414471 Compare September 12, 2026 04:29

scama added 5 commits September 12, 2026 18:21

fix(skippy): address consolidated L3 review findings

f8637d3

Preserve routable aliases for same-model workers, serialize packed GC with publication, reconcile direct store opens, and harden replay and canary validation boundaries.

Merge remote-tracking branch 'origin/scama/skippy-l3-streaming-restor…

30d0d49

…e' into jy/skippy-l2-host-tier

chore(ci): retrigger consolidated L3 validation

312b176

Merge remote-tracking branch 'origin/scama/skippy-l3-streaming-restor…

6c37716

…e' into jy/skippy-l2-host-tier

test(mesh): align review regressions with runtime contracts

15aa711

Update transitive-memory assertions for omission clearing and compare self-fill output with the public aliases that peers actually advertise.

danielwinterw mentioned this pull request Sep 12, 2026

feat(skippy-cache): benefit-per-exclusive-byte admission policy (#1650 slice 1) #1747

Closed

danielwinterw approved these changes Sep 12, 2026

View reviewed changes

danielwinterw mentioned this pull request Sep 12, 2026

feat(skippy): add durable local KV cache and restart restore #1736

Closed

Merge remote-tracking branch 'origin/scama/skippy-l3-streaming-restor…

822b8c1

…e' into jy/skippy-l2-host-tier

i386 force-pushed the scama/skippy-l3-streaming-restore branch from d0c92eb to 9550c5f Compare September 12, 2026 11:41

i386 added 2 commits September 12, 2026 22:02

chore(lockfile): include L2 cache dependency

342010a

i386 mentioned this pull request Sep 13, 2026

feat(skippy): wire next-generation KV cache and publisher defaults #1838

Draft

ndizazzo modified the milestones: 0.77.0, 0.78.0 Sep 14, 2026

i386 closed this Sep 14, 2026

Navigation Menu

Uh oh!

Uh oh!

feat(skippy-cache): bounded host-RAM L2 tier prototype (#1651) - #1749

feat(skippy-cache): bounded host-RAM L2 tier prototype (#1651)#1749
i386 wants to merge 47 commits into
scama/skippy-l3-streaming-restorefrom
jy/skippy-l2-host-tier

i386 commented Sep 10, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Sep 10, 2026

Uh oh!

coderabbitai Bot commented Sep 10, 2026 •

edited

Loading

Review skipped

Uh oh!

i386 commented Sep 10, 2026

Uh oh!

i386 commented Sep 10, 2026

Uh oh!

danielwinterw left a comment

Uh oh!

i386 commented Sep 14, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

Uh oh!

Uh oh!

Conversation

i386 commented Sep 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Bench numbers (M1 Ultra, release build, 50 pairs, head f678ce337)

Validation

Uh oh!

github-actions Bot commented Sep 10, 2026

Uh oh!

coderabbitai Bot commented Sep 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review skipped

Uh oh!

i386 commented Sep 10, 2026

Uh oh!

i386 commented Sep 10, 2026

Uh oh!

danielwinterw left a comment

Choose a reason for hiding this comment

Uh oh!

i386 commented Sep 14, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

i386 commented Sep 10, 2026 •

edited

Loading

Bench numbers (M1 Ultra, release build, 50 pairs, head `f678ce337`)

coderabbitai Bot commented Sep 10, 2026 •

edited

Loading