Conversation
…#1652) First slice of codec versioning: record an explicit codec name+version in the L3 handoff manifest so payload representations are namespaced rather than guessed, and refuse a payload this build cannot decode before assembly. - Add PayloadCodec { name, version } and CODEC_RAW / CODEC_RAW_VERSION. raw is the only implemented representation; the segment bytes are unchanged. - HandoffManifest gains a #[serde(default)] codec field, defaulting to raw. A manifest written before codec identity existed has no field and reads as raw — backward compatible, and MANIFEST_VERSION is unchanged so existing on-disk entries stay valid. - reject_unsupported_codec gates both try_commit (nothing unassemblable is ever persisted) and assemble (an unknown codec/version is a miss, never a silent misinterpretation of the bytes). Tests (skippy-cache, l3): explicit-raw round trip, legacy manifest without a codec field reads/assembles as raw, unknown codec refused at commit (no manifest left) and before assembly, unknown raw version refused on both paths, and a supported-codec payload whose bytes are corrupted still fails digest verification. No Q8/Q4 data path, request-path, or #1650/#1651 changes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…d paths (#1652 review) Addresses scama's two compatibility blockers on the raw codec slice. 1. Codec identity was downgradeable. With a serde-default codec at a fixed manifest version, the field could be stripped to force a raw reinterpretation of possibly non-raw bytes (the whole-payload digest still matches those bytes, bypassing the gate). Fix: bump MANIFEST_VERSION to 3, which *requires* an explicit codec, and add LEGACY_MANIFEST_VERSION (2) decoded as raw through an explicit legacy path. `codec` is now `Option<PayloadCodec>` so a stripped v3 codec is detectable and rejected instead of defaulted. 2. Unsupported codecs were locatable "hits". decode_manifest now centrally rejects an unsupported/absent codec, so every load path enforces it: startup reconciliation quarantines the manifest, manifest_for_prefix prunes the bad link and locate_longest falls back to a shorter supported prefix, and a direct load_manifest fails — all before segments are read or the LRU heats. Tests: downgrade (strip codec from a current-version manifest rejects; a genuine v2 legacy manifest still reads/assembles as raw), on-disk direct-load rejection, startup reconciliation quarantine, and locate_longest unsupported-longest/supported-shorter fallback — in addition to the existing round-trip, unknown-codec/version, and corruption-guard cases. Scope unchanged: payload-level raw-only; per-segment/mixed codec identity is later #1652 work. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…commit/assemble (#1652 review) try_commit and assemble checked codec support but not manifest.version, so a raw-codec manifest carrying an unknown future version could be persisted (and its segments read) even though decode_manifest/load_manifest reject it on the next read. Centralize both checks in validate_manifest_compatibility (supported current/legacy version + supported codec) and call it before commit and before any segment access. Regressions: a future-version raw manifest is refused by commit (leaving no persisted manifest) and before assembly. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bumps the L3 manifest format to v4 and moves codec identity down to the segment level, the contract #1652 needs before any non-raw codec exists: each segment names its codec, representation version, exact/lossy class, decoded length, and (lossy only) calibration digest. - MANIFEST_VERSION is now 4 and requires explicit identity on every segment; v3 (#1750, payload-level codec) and v2 (pre-codec) become explicit legacy read paths, never written. - The per-segment identity is Option only so a v4 manifest with the field stripped is detectable and rejected — it never falls back to the payload codec or to raw. The check runs in decode_manifest (load), try_commit (persist), and assemble (read), so nothing unloadable is ever persisted or partially served. - CodecClass distinguishes exact entries (payload-digest verified) from lossy ones (calibration-namespaced, never satisfy an exact lookup). A lossy identity must carry a calibration digest; an exact identity must decode to its own stored bytes. - Writers stamp SegmentCodecIdentity::raw on every segment; capability negotiation names the offending segment index on refusal. Regressions: v4 per-segment stamping round trip, v3 read compatibility, v4 stripped-segment rejection at load/commit/assemble (single and all segments), unsupported segment codec naming the segment, and decoded_len mismatch refusal. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds the deterministic CacheGen reference the later GPU work must match bit-for-bit: per-segment min/max affine 4-bit quantization, token-axis delta decorrelation, and static-CDF byte-rANS, wrapped in a versioned CGv1 container that carries its calibration and 16-entry symbol histogram (the CDF metadata) so decoding is self-contained. - reference.rs is the math oracle: every op is single-precision IEEE-754 or integer, so any backend performing the same ops in the same order reproduces it exactly. The calibration digest is BLAKE3 over the bit-exact calibration values plus tile shape — lossy lookups match only identically-calibrated entries. - rans.rs ports the canonical public-domain ryg_rans byte coder (the same one LMCache uses), deliberately in divide/mod reference form: this crate is the correctness oracle, not the performance path. - container.rs defines the wire format, the lossy SegmentCodecIdentity a CacheGen segment carries, and the store-contract test proving a real CacheGen segment is refused at commit while raw is the only supported class. Deliberate simplification, documented in reference.rs: calibration is per-segment min/max affine rather than the paper's per-model K/M-mixed calibration; the container is shape-versioned so both are follow-up experiments behind the same wire contract. skippy-protocol: the f16 RNE conversion moves to a pub(crate) module re-exported from binary, so the reference reuses the exact conversion instead of duplicating a subtle bit-exact routine. Regressions: rANS round trips (skewed, single-symbol, table rejection), quantization error bounds, flat tiles, delta wrap-around, deterministic histogram derivation, container corruption rejection (magic, reserved bytes, length lies, histogram tampering), noise non-expansion, per-segment identity namespacing, and the end-to-end store refusal. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…1652) Slice 3 and 4 of the #1652 follow-on, both evidence-only: - cubecl 0.10.0 is pinned behind the optional `cachegen-spike` feature; nothing in the library links CubeCL. The example proves bit-exact CPU+Metal parity of the quantize+delta and undelta+dequantize kernels against the CPU golden reference, and reports the six agreed metrics: cold JIT, warm dispatch, H2D/D2H bytes, peak temporary memory, encoded-size ratio (0.075 = 13.3x on the smooth 4096x128 tile), and equality (symbols and values exact on both cubecl-cpu and wgpu/Metal). - Honest kernel shape: quantization is embarrassingly parallel; the token-axis delta is a per-column sequential scan, so the result is bit-exact by construction. A parallel scan is the follow-up; rANS stays CPU in this reference and is measured as ratio only. - The stop rule is respected: the spike measured the honest copy path (create_from_slice/read_one, no zero-copy interop with the store's packed segments); the evidence goes back before CubeCL becomes a committed dependency. - docs/skippy/CACHEGEN_BACKEND_PLAN.md: Metal and CPU are the only implemented backends (real-hardware verified); CUDA and HIP/ROCm are compile-only gates until they run on real hardware, with explicit capability failure through the v4 per-segment identity gate and no hidden fallback. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
This pull request is currently a draft. Reviews will not take place until the PR is marked as ready for review. |
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Advanced Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Exact-head review at
Keep the stop rule active. The backend-plan table should describe Metal as a verified quantize/delta kernel spike until the full codec path and honest synchronized measurements pass; it currently overstates that backend as implemented. |
…ke timings, golden fixture (#1652) Address the five PR #1752 review blockers on top of 106c317: - CGv1 histogram entries widen from u16 to u32 counts. A 4096x128 tile is 524,288 symbols and cannot be encoded under the old cap; CGv1 now carries every tile the codec accepts. Regression: end-to-end encode->container->decode at exactly the measured 4096x128 shape. - The container parser validates sum(histogram) == rows*dims before any table is built, and histogram_to_freqs enforces the same caller contract, so a forged or corrupt header can no longer drive unbounded normalization repair; the convergence loop carries an explicit iteration bound. Regressions: corrupt + off-by-one histogram-total rejection at both layers. - Spike timings are now synchronized: every timed stage ends with client.sync() inside the timer (warm dispatch is total/iterations, not unsynchronized enqueue), the client parameter is used, cold JIT is the true first synchronized launch of the kernel specialization, and equality is bitwise (to_bits) with mismatch counts reported. Release-run numbers on M2 Max, 4096x128: cold 38/8 ms (CPU/Metal quantize+delta), warm 0.58/4.6 ms; ratio 0.075; 0/524288 mismatches on both backends. - An independent golden rANS stream is committed (src/cachegen/fixtures/ryg_rans_golden.bin) with a deterministic generator test (--ignored) and a decoder test that reproduces the declared symbol sequence from the frozen artifact. - cubecl is pinned to exactly =0.10.0. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…zed re-run (#1652) The backend plan's six-metric table now reflects the PR #1752 fix head: synchronized stages (client.sync() inside the timer), true first-launch cold JIT, bitwise equality with mismatch counts, and the convergence sweep across iteration counts. Notes why per-process first launch is the true cold path in cubecl 0.10.0 (the only on-disk kernel cache, SPIR-V, is Vulkan-only and not enabled here). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Fix head for all five review blockers is pushed:
1. u16 histogram cap (blocker 1): CGv1 histogram entries are now 2. Unbounded normalization (blocker 2): the parser now rejects any container where 3. Timings (blocker 3): 4. Golden fixture (blocker 4): 5. Exact pin (blocker 5): PR stays draft; the copy-path stop rule is untouched. Validation on the fix head: skippy-cache 160 passed / 2 ignored (including the fixture generator), skippy-protocol 72 passed, clippy |
|
Exact-head re-review at Three blockers remain before this leaves draft.
Validation on this exact head:
The red GitHub checks are indeed planner-catalog drift rather than a code/test failure: this head and base Keep #1752 draft and keep the copy-path stop rule active. |
… honest spike accounting (#1652) Address the three re-review blockers on top of 1d3bbd6: - Container boundary: parse_container now rejects non-finite calibration (NaN/inf min or scale), negative scale (flat-tile scale == 0 stays legal), checked-overflow tile shapes, and any rows*dims above a format ceiling (MAX_DECODED_VALUES = 2^30). The validated count travels in the header; decoded_value_count, container_calibration, and decode_f16_segment all refuse, and decode sizes (count*2) are checked before allocation. Regressions reproduce the review's exact attacks: NaN min, inf min, NaN scale, scale = -1 (all three consumers), and the 92-byte rows=u32::MAX container that used to report 68,719,476,720 values. - Golden fixture now pins the encoder too: a normal test asserts the Rust encoder reproduces the frozen 129-byte stream byte-for-byte for the pinned symbols/CDF (upstream ryg_rans c9d162d9, stream SHA-256 7887612d...), so encoder drift fails CI instead of being overwritten by the same-implementation generator. - Spike accounting: live device buffer peak now sums all four buffers the harness holds (tile + calibration + symbols + rebuilt = 6,291,464 bytes on CPU, 6,291,712 on Metal), H2D includes the 8-byte calibration upload, and H2D/D2H are timed to completion (CPU: 2,097,160 B up / 4,194,304 B down; Metal: ~0.9-1.4 ms up, ~3.0-4.1 ms down). Backend plan relabels CPU/Metal as quantize+delta kernel-spike verified, full CacheGen backend unimplemented, rANS still CPU. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Fix head for the three re-review blockers is pushed: 1. Container boundary (blocker 1): 2. Encoder pin (blocker 2): new normal (non-ignored) test 3. Spike accounting (blocker 3): the peak now sums all four live buffers — tile + calibration + symbols + rebuilt = 6,291,464 bytes on CPU, 6,291,712 on Metal (allocator rounding difference visible and reported) — H2D includes the 8-byte calibration upload (2,097,160 bytes total), and H2D/D2H are timed to completion: CPU 2,097,160 B up in ~12–18 ms / 4,194,304 B down in ~36–44 µs; Metal up in ~0.9–1.4 ms, down in ~3.0–4.1 ms. Stack handling per your instruction: no rebase yet — this head still carries catalog blob Validation on |
|
Exact-head re-review at Blocking: the inclusive format ceiling still permits a multi-GiB allocation bomb
Please bind the decoder to the segment's expected Blocking validation failureBoth clippy configurations fail under the unpinned current Rust 1.98.0 toolchain at This is confined to the new hostile-header regression and should be mechanical to fix, but Cleared from the prior review
Exact-head verification:
Keep the PR draft and leave the stack unsynced for now. Once this boundary is made operationally safe and clippy is green, sync the dependent stack once against current main to clear the known slice-catalog planner failure. |
…orced symmetrically (#1652) Third re-review blocker plus the Rust 1.98 clippy failure: - MAX_DECODED_VALUES drops from 2^30 to 2^24 (16,777,216 values = 32 MiB decoded f16). Decode working set is ~7 bytes per value (symbols + f32 materialization + f16 output), so the worst case a header can command falls from ~7 GiB to ~112 MiB; the measured 4096x128 tile sits 32x under the ceiling. Raising it is a format-version decision, not a per-parse judgment call. - The ceiling is now one shared admission check (checked_tile_len) routed through both parse_container and encode_f16_segment, so encode and decode are symmetric by construction. The encode guard runs before the f32 materialization: an over-ceiling input is refused without doing the work it names. - Boundary regressions: a crafted 65536x256 header is admitted at exactly MAX_DECODED_VALUES; 16777217 == 97 * 172961 (a real shape, not a truncation artifact) is refused one value later by decoded_value_count, container_calibration, and decode_f16_segment, naming the ceiling. The encode guard is probed at MAX and MAX + 1 without allocating the tile, plus an end-to-end refusal of a real 33.5 MB input. - Clippy on unpinned Rust 1.98.0: the bomb test's chunks_exact_mut loop becomes a byte fill (LE u32::MAX is 0xFF bytes) and the new helper writes its histogram the way the encoder does. - Console-print ratchet regenerated: allowlist entries had drifted under the stack (l3/tests.rs print moved 605 -> 608 in the manifest-v4 commit; the cachegen_cubecl_spike example landed in round 1 without a ratchet run). Mechanical only - the print moved, and the example's existing prints were never allowlisted. Validation on this head: fmt clean; skippy-cache 165 passed / 2 ignored; skippy-protocol 72 passed; clippy -D warnings --all-targets with and without cachegen-spike; release spike PASS on CPU and Metal (0/524288 mismatches, ratio 0.075, live peak 6291464/6291712 bytes); no-console-print and repo-consistency pass.
|
Fix head for the remaining allocation-safety blocker and the Rust 1.98 clippy failure. Exact head: 1. Working-set ceiling, not a token-count ceiling — 2. Symmetric by construction — the ceiling is one shared admission check, 3. Boundary tested at both edges — a crafted header at exactly 65,536×256 is admitted ( 4. Clippy 1.98 — the bomb test's 5. One honest find on top — running the repo's full pre-push ritual for the first time in this stack surfaced console-print ratchet drift: the Validation on |
|
Exact-head re-review of The remaining allocation-safety blocker is cleared. I independently reproduced on the exact head:
No further code blocker found in this fix. The PR should remain draft for the one-time dependent-stack sync: current exact-head checks still fail only at planning with |
|
Post-sync review of The sync itself is sound:
The current GitHub result is not yet the full post-sync matrix. All five workflow runs are terminal and their planners pass, but the PR is still draft, so the plan selected Therefore the merge-sync review is cleared, but CI execution evidence is still outstanding. Marking the PR ready should trigger the |
…s and fixture (#1652) Address the five PR #1752 review blockers on top of 106c317: - CGv1 histogram entries widen from u16 to u32 counts. A 4096x128 tile is 524,288 symbols and cannot be encoded under the old cap; CGv1 now carries every tile the codec accepts. Regression: end-to-end encode->container->decode at exactly the measured 4096x128 shape. - The container parser validates sum(histogram) == rows*dims before any table is built, and histogram_to_freqs enforces the same caller contract, so a forged or corrupt header can no longer drive unbounded normalization repair; the convergence loop carries an explicit iteration bound. Regressions: corrupt + off-by-one histogram-total rejection at both layers. - Spike timings are now synchronized: every timed stage ends with client.sync() inside the timer (warm dispatch is total/iterations, not unsynchronized enqueue), the client parameter is used, cold JIT is the true first synchronized launch of the kernel specialization, and equality is bitwise (to_bits) with mismatch counts reported. Release-run numbers on M2 Max, 4096x128: cold 38/8 ms (CPU/Metal quantize+delta), warm 0.58/4.6 ms; ratio 0.075; 0/524288 mismatches on both backends. - An independent golden rANS stream is committed (src/cachegen/fixtures/ryg_rans_golden.bin) with a deterministic generator test (--ignored) and a decoder test that reproduces the declared symbol sequence from the frozen artifact. - cubecl is pinned to exactly =0.10.0.
…zed re-run (#1652) The backend plan's six-metric table now reflects the PR #1752 fix head: synchronized stages (client.sync() inside the timer), true first-launch cold JIT, bitwise equality with mismatch counts, and the convergence sweep across iteration counts. Notes why per-process first launch is the true cold path in cubecl 0.10.0 (the only on-disk kernel cache, SPIR-V, is Vulkan-only and not enabled here).
First reviewable follow-on slice of #1652, stacked on #1750 (head
a0876ee0) over #1736 (heade63f86ca6). Synced onto maind4ffbbacd2026-09-10 by ordered merge-forward — reviewed heads7247a397a/5effeffb1/34dd77e9call preserved as ancestors; the only manual conflict resolutions were two Cargo.toml dep blocks (main's v0.76.0 version bump vs the stack's added deps).29d87ef3c— manifest v4: per-segment codec capability contract. Every v4 segment carries{codec, version, exact/lossy class, decoded_len, calibration_digest}. Stripped-field v4 rejects at load, commit, and assembly (no payload-codec/raw fallback); v2/v3 read-compat on-disk; unsupported segment codecs refused naming the segment index.CodecClassseparates exact (payload-digest verified) from lossy (calibration-namespaced, never satisfies an exact lookup).23814d0ad— pure-Rust CPU golden CacheGen. Static-CDF byte-rANS ported from public-domainryg_rans(the coder LMCache ships), per-segment min/max affine 4-bit quantization, mod-16 token-axis delta, CGv1 container carrying calibration + 16-entry CDF histogram. Calibration digest (BLAKE3 over bit-exact calibration + shape) namespaces lossy entries. Test proves a real CacheGen segment is refused at commit while raw is the only supported class. skippy-protocol's f16 RNE conversion is now shared, not duplicated.106c31752— CubeCL feasibility spike + backend plan.cubecl 0.10.0pinned behind the optionalcachegen-spikefeature; nothing in the library links CubeCL (now pinned=0.10.0).e00a539f4+1d3bbd672— first-round review fixes: CGv1 u32 histogram + header validation, synchronized spike timings, committed golden fixture; backend plan re-measured with the synchronized harness.03d1fbd2a— second-round review fixes: container boundary validation (NaN/inf calibration, negative scale — flat-tile scale 0 stays legal — overflowed shapes, with attack-reproduction regressions); the golden fixture now pins the encoder too (byte-for-byte vs the frozen 129-byte stream, provenance upstreamc9d162d9→ independent C repro → fixture); honest spike accounting (all four live buffers, H2D includes the 8-byte calibration, H2D/D2H timed to completion).34dd77e9c— third-round review fix:MAX_DECODED_VALUESbound to a decode working set, not a token count —2^30→2^24(32 MiB decoded f16; worst-case decode working set ~112 MiB vs the ~7 GiB a 92-byte header could previously command), enforced symmetrically through one shared admission check (checked_tile_len) routed through both parse and encode, encode guard firing before the f32 materialization. Boundary regressions at both edges (65536×256 admitted; 16777217 = 97×172961 refused everywhere); Rust 1.98 clippy fixed; console-print allowlist regenerated for the stack's drift.Spike results (M2 Max, 4096x128 tile, release build, synchronized stages, bitwise equality)
Every timed stage ends with
client.sync()inside the timer; H2D/D2H are timed to real completion. Honest shape: quantization is embarrassingly parallel; the token-axis delta is a per-column sequential scan so output is bit-exact by construction (parallel scan = follow-up). rANS stays CPU in this reference, measured as ratio only. Copy path is the honest one (create_from_slice/read_one, no zero-copy into the store's packed segments) — per the agreed stop rule, evidence is back before CubeCL becomes a committed dependency.Backends: CPU and Metal are the only implemented ones (real-hardware verified). CUDA and HIP/ROCm are compile-only gates until they run on real hardware —
docs/skippy/CACHEGEN_BACKEND_PLAN.md(relabeled: quantize+delta kernel spike verified; full backend unimplemented).Bounds held: no request-path wiring, no Candle/Burn/Python/PyTorch, native runtime-format passthrough remains the exact control arm, lossy entries can never satisfy an exact lookup.
Status
34dd77e9c; post-sync review cleared ataa533c0d4(merge chain, ancestor preservation, conflict resolutions, both CI catalog blobs identical to main).pr-draftprofile,required_slices=[]; execution lanes skipped — correct for a draft). The full execution gate is the outstanding item.aa533c0d45e1b8af025ee9af9887de68a7f02baf, tree clean: skippy-cache 165 passed / 2 ignored, skippy-protocol 72 passed, fmt clean, clippy-D warnings --all-targetswith and withoutcachegen-spike,no-console-print+ repo-consistency pass, release spike PASS CPU+Metal.pr-readymatrix, or trigger the CI Manual Full workflow (workflow_dispatch) while staying draft. Draft state unchanged; CacheGen stop rule intact.