[gilby]

Fixes #1360.

--clef DIR serves Cloudflare Clef decisions on /v1/systemone: the record encoded as Cloudflare's encode_record does, the engine's final rows for the prompt, and Clef's joint schema head on the host. DIR holds the release's joint_head.safetensors and joint_head_config.json, and lm_head_rows.safetensors (the backbone's output embedding, which the head reads for each option's lexical prior).

Commits

1. clef: Cloudflare Clef's record encoder and joint schema head on the host: equal to its Python within fp32 rounding (golden test on three records: token ids and question and option spans identical, logits within 1.5e-6, probabilities within 3.4e-7).
2. clef: --clef DIR answers /v1/systemone through Clef's head: a hidden engine call returns every prompt row's final normed hidden state; the lane host queues it like the decision calls; the Qwen3.5 CUDA family implements it. Chat is untouched when the flag is absent.
3. clef: "raw": true: each question's unrounded option probabilities in encoder order, for gates.
4. clef: the head's linear rows in blocks of 8 and its attention split across threads: a weight row is read once a block of rows, and every output is the same dot as before, so the bits are unchanged; 1.4–1.9× faster decisions.
5. clef: Cloudflare's Clef code in THIRD_PARTY_NOTICES; the refusal names the qwen3_5 CUDA family.
6. clef: a decision's prompt steps one max_rows piece per engine-thread pass, on a cache reserved at startup: hidden(prompt, out, at) returns the next row; the score queue runs one piece of the oldest decision per pass and goes back to chat rounds. The decision cache (16,384 rows) is created in the family's open, inside the memory budget.
7. clef: a tiny random head against Cloudflare's JointSchemaHead, and 1 vs 4 head threads bit-equal, in CI: tools/clef_tiny_golden.py writes zig/src/server/fixtures/clef_tiny.json from Cloudflare's own code; clef_test.zig checks logits within 1e-4 and 1 vs 4 threads bit-identical.

Receipts (one DGX Spark, GB10 128 GB, driver 580, CUDA 13.0; Cloudflare/clef-flash bf16)

The checkpoint loads unchanged with "quantization_config": {"quant_method": "compressed-tensors"} added to its config.json (no packed or scaled weights, so every projection is bf16). Plain bf16 loading is #1137's.

• Fidelity vs Cloudflare's own bf16 pipeline (load_release_model, transformers 5.10.2) on 60 records / 557 questions: mean KL 5.7e-5 (max 2.4e-3), top choice 99.64%. ≤1K 95.5% (1 of 22), 1–4K 100%, 4–10K 99.6%, >10K 100%.

• Exactness: 4 concurrent requests return exactly one-at-a-time probabilities for all 60 records; bench_concurrent --tokens 128 --reps 3 --temperatures 0 --levels 1,4 --alone --serial: 0 unequal, 0 failed.

• Latency, one request at a time, prompt cache off, p50 / p95:

  record	before commit 4	this PR
  ≤1K	0.49 / 0.83 s	0.35 / 0.60 s
  1–4K	1.75 / 3.42 s	1.22 / 2.04 s
  4–10K	6.00 / 11.58 s	4.29 / 5.49 s
  >10K	16.33 / 21.28 s	8.80 / 9.67 s

  A cold 16K prompt pass alone takes 6.0 s (~2,660 tok/s); the rest of a 16K decision is the host head.

• Chat during a decision (commit 6): a greedy chat stream's longest gap between tokens while one 16,384-token decision runs: 6.31 s before, 1.69 s now (one 4,096-row piece); median gap 69–71 ms either way, 70 ms with no decision. Decisions are unchanged (60/60 records equal, KL and top choice as above, re-run on the head).

• zig build test --summary all (macOS): 500/500 steps, including the tiny-head golden and the 1 vs 4 threads test. zig fmt clean; lean_check.py adds no problems over main. ATC overlap: no textual conflicts added against open PRs (qwen27: a JointHead side-head scores options on post-norm hidden rows at their final-token positions #1191, lanes: a queued decision takes a free slot beside the live streams, as a new stream's prompt pass would #1192 and qwen27: TF_Q27_PROFILE logs the round's class GPU times, and replies carry gpu_seconds #1215 also touch systemone.zig and cli.zig).

The 27B Cloudflare/clef needs its adapter at runtime rather than merged into 4-bit weights (see #1360); not in this PR.

[gilby]

For the Qwen3.8-27B Clef: its runtime adapter (#1364) loads Clef as a LoRA on the base checkpoint. Cloudflare publishes Clef only as merged weights, so the extracted adapter (rank-256, layers 40–63; base + adapter vs the merged release: mean KL 3.9e-7, top choice 100%) plus Cloudflare's joint head files is published at unigilby/Qwen3.8-27B-clef-adapter (Apache-2.0). Clef-Flash itself (this PR) needs nothing extra: it loads Cloudflare/clef-flash as released.

[ashhart]

Thanks @gilby. --clef is off by default and /v1/systemone is unchanged without it, which is right. Before device runs:

1. The port of Cloudflare's code (clef_encode.zig, clef_head.zig, clef_serve.zig, the verbatim system prompt) needs a THIRD_PARTY_NOTICES entry for its Apache-2.0 source.
2. With --clef on, hiddenFn runs a decision's whole prompt (up to 16k rows) in one go on the engine thread, which stalls every chat stream for seconds (about 6 s on the 9B). Please step it one max_rows piece per round like other prompt work.
3. E.Cache.init per decision allocates outside the memory budget; reserve it at startup.
4. The refusal text names Qwen3.8-27B, but the engine is the qwen3_5 CUDA family.
5. Please add a CI-runnable test of the head (a tiny synthetic golden) and of the "bits unchanged across threads" claim (1 vs N threads).

[gilby]

Thanks for the review. All five points are addressed in three new commits on top of the old head (63fb8f13a → cce6cbeb5, fast-forward):

1. Notices: THIRD_PARTY_NOTICES.md has a new "Cloudflare Clef" section. It names the ported files (clef_encode.zig, clef_head.zig, clef_serve.zig with the verbatim system prompt) and the functions they follow in joint_schema_model.py, with Copyright Cloudflare, Inc., Apache-2.0, LICENSES/Apache-2.0.txt. (c6a7a4a8c)
2. Stall: hidden is now a stepped backend call, hidden(prompt, out, at) -> next row. The lane host's score queue runs one max_rows piece of the oldest decision per engine-thread pass, then returns to chat rounds. The idle wait in lane_host.zig and lane_rounds.zig stays awake while a decision is in progress. The pieces are the same pieces in the same order as the old one-shot loop, so decisions are unchanged: 60/60 records give equal probabilities, and 4 concurrent decisions equal one at a time. A host test checks that a 9-row prompt on a 3-rows-per-piece fake backend is called at rows 0, 3 and 6 and matches a whole pass. On Clef-Flash (spark2, 4096-row passes) I measured a greedy chat stream's longest gap between tokens while one 16,384-token decision runs: 6.31 s before, 1.69 s now (one piece). The median gap is unchanged at 69–71 ms, and 70 ms with no decision running. (a2e1d23d4)
3. Budget: the decision cache (E.Cache, decision_rows = 16384 when --clef is given) is created and touched inside the family's open, so the native host's model measurement counts it. Nothing is allocated per decision. hiddenFn refuses a prompt longer than the reserved cache. (a2e1d23d4)
4. Refusal text: it now reads "Clef needs the qwen3_5 CUDA family's native engine". (c6a7a4a8c)
5. CI tests: tools/clef_tiny_golden.py builds a tiny random JointSchemaHead with Cloudflare's own code (H 16, width 8, 1 routing and 1 decoder layer, 2 heads, vocab 20, 12 tokens, two questions of kinds 1 and 2). Its weights are rounded to bf16, and it writes zig/src/server/fixtures/clef_tiny.json (24 KB). clef_test.zig checks the Zig head's logits against it within 1e-4, and checks that the head with 1 thread and with 4 threads gives bit-identical logits (Config.threads caps the head's threads). Both run in the plain zig build test; I broke each once and confirmed it fails. (cce6cbeb5)

Gates on this head (one DGX Spark, GB10, CUDA 13.0, -Dsm=121, Cloudflare/clef-flash bf16, Cloudflare's 60 records / 557 questions):

• Mean KL 5.66e-5 and top choice 99.64%, the same as before the change. Gate passes.
• 4 concurrent decisions equal decisions one at a time: 60/60 records.
• bench_concurrent --tokens 128 --reps 3 --temperatures 0 --levels 1,4 --alone --serial: 0 unequal, 0 failed.
• zig build test: 500/500 steps. zig fmt is clean, and lean_check.py adds no problems.

The 27B through #1364 (rebased on this head) also passes: see there.

[ashhart]

Review of the Clef set (#1361, #1364, #1366, #1372) against main e9b69ed. Thank you, @gilby: Clef on /v1/systemone is a good fit, and --clef being off by default with chat untouched is exactly right. Notes on #1361 here; the adapter PRs have their own.

1. Head reader. Files.open reads eight bytes without a length check and then trusts the header length (server/clef_head.zig:22): a four-byte file aborts at line 26. Files.get unwraps JSON kinds and trusts dimensions and offsets (:35, :253). Please use the shared bounded safetensors reader, check dtype, every dimension, byte extent and overflow before slicing or allocating, and add malformed-header, truncated, wrong-shape and allocation-failure tests. The head config should also refuse a width not divisible by the head count (clef_serve.zig:34), and Clef.open needs an errdefer c.head.deinit() after the head loads (:43).
2. Shared rounds. lane_host_score.answer runs each ask's whole prompt through a fresh cache in turn and marks them done only at the end (core/lane_host_score.zig:59–:96; qwen3_5/cuda_native.zig:357), so decisions run serially and no decode round happens between their chunks. Please admit decision prompts into the shared scheduler (row ownership, stepping, cancellation), keeping solo results byte-exact.
3. Rounding. round4 doesn't match Python's rounding (clef_serve.zig:143): 0.03125 gives 0.0313 here and 0.0312 in Python. Please add release-compatible rounding with boundary tests.
4. Reference test. The golden test hardcodes the 5120-wide head and asserts only probability error. Hermetic encoder/head fixtures with option spans, truncation edges, non-finite inputs and one- vs many-thread equality would let it run in CI, parameterised for Clef-Flash's width too.

[ashhart]

Thanks @gilby for Clef on /v1/systemone. We're closing this for now, as our 26 Oct review hasn't had a reply. Please reopen it once the head reader uses the shared bounded safetensors reader and decision prompts join the shared rounds.