Repository navigation
lfm2: keep a drafter's tapped rows on every forward - #1390
Conversation
ashhart
left a comment
ashhart
left a comment
There was a problem hiding this comment.
Thanks @BlivionIaG. The tap operator has the same contract gap as #1377 (no planted controls or recorded launch limits), and this stacks on #1377.
|
Thanks @BlivionIaG. The tap copy now has its launch record (M1 and M2 from It merges cleanly onto current main, and fmt, the lean check and |
3df2d05 to
4156e9e
Compare
|
@ashhart Rebased the tap-only branch on current main |
The lane core can put an external drafter in front of a target. The drafter reads the target's hidden rows after some layers (its taps). LFM2-MoE had no taps, so it could draft only with the core's own heads. prepare_features names the layers. From then on, each forward copies each row's output of those layers into the scratch (lfm2_tap, a plain copy of bf16 values). features() gives a stream's rows of the latest verify or one-token round. prefill_features hands each prompt chunk's rows to the drafter before the chunk's end is kept for reuse. The prompt's last row is kept beside the stream's caches for its first draft. A drafting stream resumes only a kept state that also holds its drafter's context (Snap.draft). Without a drafter nothing changes: no taps are kept and every forward runs as before. lfm2_tap registers with the kernel contract (metal_lfm2_tap.zig): the third of five taps' columns at 1, 10, 32 and 2,048 rows must hold the layer's bits and the other columns what the call found. Gate 5 checks its launch against every Apple chip we serve and this GPU. test-lfm2-dspark checks the same copy through the family's own kernels.
…planted controls lfm2_tap's launch is in lfm2_records.zig with the limits --offline compiled, not this GPU's. Two controls edit the source: rows packed at the hidden size, and a column of another tap overwritten. Each goes red at the shapes gate.
4156e9e to
5a10ade
Compare
|
Thank you @BlivionIaG. This is in TensorFold 1.2.0, released today: https://github.com/ashhart/TensorFold/releases/tag/v1.2.0 It landed with your commits under your own name, so you're listed as a TensorFold contributor, and it's credited to you in the release notes and the changelog. |
Keeps a drafter’s tapped target rows across forward, prefill and retained-state paths. Without a drafter the tap path remains off.
Rebuilt on main
d33dcfc767ee093393c6bfa00b3949bd7b4d8e88. The rejected #1379/#1382 guard commits are excluded. No M5 was accessed for this restack. Fresh CI is running; maintainer re-review remains required.Scope and review fixes
Standalone tap changes only; independent of the pricing PR. Tap launch records and two planted controls are retained, along with current main’s unrelated contract registrations.
Validation
Head
4156e9e62. 57/57 build steps and 11/11 Metal draft-source host tests pass; the kernel-contract runner compiles. Lean: 425 on main and this branch, zero new. Device evidence in older comments remains scoped to its original head.Part of #1374. The chain’s intermediate branches are now updated rather than leaving fixes only in the cumulative tip.