Repository navigation
upstream の v1.1.1 を取り込み、GLM CUDA を upstream のエンジンへ切り替える（3bit EXL3 は維持） - #3
Merged
Merged
Conversation
… other ranks end their job loops deinit sent the TP worker its end message, a memset and an NCCL send on the engine's stream, before closeEntry made the engine's context current, so a close from a thread with another context current queued them in the wrong one. The native wrapper rebinds before it closes; a direct close now gets the same order: closeEntry first, then the worker's end. No host fake drives a TP worker's end; the two-rank Super gates cover this path.
…ame off macOS The load and recovery fixtures gave their fake driver /usr/lib/libSystem.B.dylib as its library, which only macOS has, so test-bolt-load-host could not start on the Linux hosts that run the CUDA gates. They open libc.so.6 there, as the copy-source race fixtures do.
…he reserved video token shielded in user text ashhart#940 samples a video's frames into groups the RADIO tower reads as images, each with its time and the words before it, and budgets video tokens apart from image tokens. It joins our vision offer beside the Qwen, Flash Next and GLM towers: Nemotron Omni's video comes from its own prepare, bounded by the serve flags' video and frame counts, and the template's `<video>` becomes the reserved token before encoding. With it, our checks: the video config, frame rate, frame indices, dimensions and grid are refused before any decode or cast, prompt spans and row products before any device use, and every owner unwinds on a failed allocation. The video types stay in engine_vision.zig with ashhart#940's labels and video text added; its video_offer.zig is not taken. Test names lose their review labels and the video step's filter becomes optional.
cut computed a span's first buffer row as sp.row + from - sp.at, left to right, so a span placed near the top of the u32 prompt range overflowed before the subtraction brought it back: at 0xffffff00 with 128 rows at buffer row 256, a pass from 0xffffff00 added 256 first. It subtracts first now. The fixture cuts that span at its start, its middle and past its end.
native/cuda.zig's memory pool and its weight accounting move to cuda_family_memory.zig, whose unused weights helper the accounting replaces, and experts.zig's row experts move to experts_rows.zig, re-exported where callers find them. No behaviour changes.
Bolt's IQ2_XXS experts read their grid and sign tables from zig/kernels/cuda/ggml_iq2xxs_tables.cuh, which holds them as ggml-common.h defines them and names ggml's MIT notice on its first line; the notices now list the file with ggml's copyright, under the MIT text they already carry, and say that the kernels using it are TensorFold's.
…496 experts take a wide one The Bolt merge grew the router top-k's experts a lane holds from 8 to 16 for every Nemotron, so the released kernel went from 91 to 95 registers to 168 on every SM and its launch records went stale, the 13.3 one with no compiler here to measure it. The body is a template now: tf_nemo_topk keeps 8 a lane, up to 256 experts and 8 picks, compiled as the release compiled it, and tf_nemo_topk_wide holds 16, up to 512 experts and 16 picks, which the glue launches past 256 experts or 8 picks. Both select the same experts for a router both can hold. The wide kernel's record is nvcc 13.0's, 168 registers on sm_80 to sm_121.
…latent or checkpoint kernels The fixtures build an Engine in undefined memory and set the fields its close reads, but not `tp`, `k.latent` or `k.ck`. A debug build fills that memory with 0xaa, which the aarch64 build read as null and the x86_64 build as present, so on x86_64 the close path tore down a TP group at a garbage address and seven tests crashed in each of the two test binaries that hold them.
nvcc 13.2 compiles tf_nemo_topk_wide to 160 registers on sm_120 and sm_121, where 13.0 gives 168; the older SMs keep 168. nvcc 13.3 takes this record until its own is measured.
…els, DFlash drafts, images and structured output ashhart#879 serves Gemma 4 natively on Metal: the 31B's config and geometry, decode glue and prompt attention on the tensor units, prompts in 512-row chunks, DFlash drafts with the drafted wrapper handing held drafts to the grammar, the 27-layer image tower and an engine-named image part type, its buffers in one residency set the idle keepalive holds, and the third-party notices for its sources. ashhart#879 changed the shared affine_mm kernel, its contract formula and lfm2_moe's callers to compute as Gemma wants; with this merge those stay as the release has them, and Gemma's arithmetic runs in its own kernel copies, so no other family's bits move. Against this branch: the drafted wrapper forwards first_before_draft beside ashhart#879's held drafts, the drafted tests keep wrapped_early beside ashhart#879's grammar cases, and the image part type joins our vision types in engine_vision.zig.
…mbedder, and the MTP assistant as a drafter ashhart#975 makes Gemma 4's sizes a runtime geometry, so the 12B (gemma4_unified, MLX 4-bit g64) runs on the same native host as the 31B: its checkpoint from Google's release, its image embedder on the tower's GEMMs with 48-pixel merged patches, and its MTP assistant as a native drafter, admitted by name and made 4-bit at load; --drafter takes DFlash or the assistant by the drafter's model type. Rebased onto the merge above, Gemma's private arithmetic and the release's shared paths stay as they are.
…sts, and the docs name Gemma's own arithmetic parse cast max_position_embeddings to a usize before its range check, and each eos id to u32 with no check, so a negative or oversized value panicked in a safe build; an eos_token_id of another JSON kind was skipped without a word. Each is checked first now, with BadGemmaContext and BadGemmaEos, and fixtures cover the boundaries. The comments and the recipe say that Gemma's prompt projections run on its own rounded-weight kernel copies, and that the PR's measurements describe its tree, not this one.
…very failure; the contract checks its kernels packArrays now reserves both owned-list slots before it publishes either packed buffer, so a failed second append no longer releases one buffer twice, and the late constructor arms a cleanup after each private pipeline and inverse table, so a failure releases every acquired handle once and none before ownership moves. Gemma's rounded dense and fused kernels join the standard Metal contract runner as a family: twelve records with numerical hooks, four planted controls and launch limits, missing or duplicated coverage refused, and hardware without tensor units reported as TensorUnitsRequired. Host fakes cover the ownership boundaries and the registration; no shader runs.
…without review labels
The shared-affine, owner and contract fixtures at zig/ took review numbers as names (t52_*, t59_*), and their tests
began with them. They are gemma4_shared_affine_test, gemma4_host_compile, gemma4_recording_metal, gemma4_inert_metal,
gemma4_owners_test and gemma4_contract_{compile,family,test} now, the test names start with what they check, and
the affine host step's filter becomes optional.
… stay out of the tree tools/zig/gemma4_gate.py and gemma4_dump.py run the Python Gemma engine the gates compare against and import it, and Python tooling that imports an engine does not land; they stay on ashhart#879's branch. The recipes, the build's step words and the Gemma test programs' headers now name the oracle and its dumped fixtures instead of the scripts.
…at has no device Gemma's shared-affine test records core/affine_mm.zig's dispatch through a recording Metal stand-in, and importing the file brings its tests along; three of them open a device first, and the stand-in has no Device.init, so the test module did not compile. They now skip at compile time when the Metal module has no Device.init, as they skip at run time without a GPU.
…afts, images and kept prompt states The merged commits keep their author's name, dates and messages; the five that carried BobClawblaw's login-only noreply address carry his id+login noreply instead. The engine API's image and video types stay in engine_vision.zig, which takes ashhart#903's image pieces and Piece; the template's part type came with Gemma 4 the same way. ashhart#903's engine_media.zig, the same types beside them, does not land. Beside Gemma 4, the hub, the Metal registry, the --vision refusal, the one-Mac and --learn names and the docs list both families.
…odel or device work Step-5's config is refused (UnsupportedStep5Experts) unless its experts are positive, a multiple of four and at most 512, its top-k is positive, at most the expert count and at most 16, its hidden width follows the router's 64-column rule, its MLP widths the indexed projection's input alignment, and its shared expert fits the scratch ashhart#903 sizes by the intermediate width. The affine loader checks each axis of weight, scales and biases, the expert stack included, with the rank before any index, the exact byte extent, checked products and the packed width, where it accepted equal element counts; router metadata must be [experts, hidden]. Host tests on serialized configs and real tensor metadata, and a macOS host step that builds them.
…ecked before a library or buffer is made The fixed projector's 4096-wide language output must match the text model, and every one of the tower's 667 required tensor records its exact shape and byte extent (norms, matrices, convolution axes, the position table, the projection) before the tower creates a library, pipelines or GPU buffers; malformed geometry returns UnexpectedStep5Tensor or UnsupportedStep5Tower. Flattened patch rows and a singleton-batch position table stay admitted, their layouts being the same bytes. Each record is tested with a wrong rank, with same-count transposes, odd byte extents and compatible layouts; an invalid load returns before an undefined device is touched.
…through an explicit pointer fitCache took the store as &(h.cache orelse return), which on Zig 0.17 does reach the owned store; the callback now captures it as `if (h.cache) |*owned|`, so no reading of the expression can make it a copy. Fixtures on the real Store, with host-memory snapshots and no GPU, cover budget clamping, held, entry and eviction counts that persist, a restoring state that stays protected, repeated calls, complete eviction and a host without a cache.
…dded them The Step-5 host step ran only tests whose names began with a review number; they begin with "step5 host:" now and the step selects that prefix, so the same fixtures run and the names carry no review label.
…hart#950, ashhart#959 and ashhart#960 beneath it The H3 and FastH3 transformer on Metal as a test tool beside the release's, clips that start from an image, any step count with its schedule and AdaLN tables, the released MiniMax-H3 weights with re-voicing of a finished clip, and Ref2VA's reference pictures, videos and audio. Against this branch: build.zig runs the H3 schedule's host tests beside the XPU admission tests.
… checked before any shader or scratch The H3 Metal tool read its count vector, held frames and audio, tiles and checkpoint dimensions where it used them. They are checked first now, each with its own refusal: six to eight counts, none negative; held rows that fit the prefix packing and held audio within the clip; tiles 1 to 64 rows summing to the rows, slots unique and in their partition; checkpoint ranks before dimension reads, fused QKV divisibility, GEMM widths divisible by 256, rotary widths 2 to 128; schedule lengths, finite timesteps and every forward slice before indexing; buffer sizes in usize and the signed Metal offset range. At video timestep 1 a held frame's 0.999 entry was missing from the sorted table and the lookup trapped; the schedule inserts it. Host fixtures for each refusal, run in the normal host suite.
…e complete A reschedule published each table as it finished it, so a failure part-way left a schedule that was half old and half new. It now allocates the steps and temporary tables, finishes every projection and command, then replaces every live table and its metadata together, releasing the old GPU tables only after completion. A host fault fixture fails the steps, a late block, the final projection and the completion in turn, then retries, and checks the old state and that each temporary owner is released; it executes no Metal command.
…n only when the file changes Checkpoint content digests are cached under HOME/.cache/tensorfold/file-digests/v1, as the cost cache keeps its tables: a bounded, checksummed, versioned index of at most 1,024 records, each keyed by device, inode, size, mtime and ctime to the nanosecond (Linux reads them with statx and refuses missing fields; macOS with fstat). A hit must match every key field, and the descriptor and path are checked again; a miss and every finished read recheck the source. Writers hold a process mutex and an advisory lock on a persistent inode and publish atomically; a corrupt, truncated, duplicated or foreign index is ignored. Fixtures cover forced rereads, a rewrite with mtime restored, corruption, a change during a read, eviction and concurrent processes.
…, MTP drafts chained in one command buffer (ashhart#606) By default Flash Next now serves on shared sessions whose count and contexts follow the memory the load leaves (FLASHNEXT_SLOT_CONTEXTS sizes them by hand): a lone request runs on the engine's own rounds and joins the shared rounds when a second arrives, a stream that outgrows its session moves to a larger one, and a move that fails ends its stream with an error. Shared rounds are priced from forwards timed at load, every stream's drafts go in one command buffer, a request's temperature draws on the host by a counting sort of bf16 keys, and FLASHNEXT_LOAD_LIMIT_GB covers the prompt cache too. Chained batch MTP drafts are experimental. When the lane core releases a stream whose release ends another, it sweeps the live list in that step, before a job can be freed. The branch's commits come in as this one, their authors credited below. Against this tree: the sweep sits in the lane core's handover.zig beside discard, the overhead timing keeps its timed switch and subtracts the priced drafts, the new lane tests are in engine_sweep_test.zig, and the failed-round path logs before it sweeps. Co-authored-by: Seth For Privacy <40500387+sethforprivacy@users.noreply.github.com> Co-authored-by: BobClawblaw <288991054+BobClawblaw@users.noreply.github.com>
…size themselves With --prompt-cache-gib unset the shared host reserved no cache and planned one after its extra sessions, where the single host takes up to 16 GiB after the footprint and a 2 GiB margin. Both now size the cache first with the same rule (budgetUnder, its arithmetic moved unchanged into cache_budget.zig) and keep that budget: the sessions get what is left, the backend's overhead included, and after the backend starts the measured footprint, margin and reserve must fit, so the cache is never shrunk to a new remainder. An explicit value, zero included, stays as given, and an explicit --prompt-cache-over-cap keeps its opt-out. Pure tests for the reserve and a compile check of the host.
The shared-session budget tests began with a review number; their names now start with what they check.
…e matmul reducing its split K through slice partials Cards without thread-block clusters (sm_80 to sm_89) now serve Nemotron's FP8 and NVFP4 ModelOpt checkpoints: the lane matmul writes its split-K sums as fp32 slice partials and reduces them, each prompt segment through its own buffer's partials, and the loader packs and uploads with scratch-free ops before the buffers exist. The merged commits keep their author's name, dates and messages; the three with BobClawblaw's login-only noreply carry his id+login noreply. Against this tree: the loader's ops go into our partial-load custody path, the lane part joins our scratch list, and the ModelOpt test keeps the partials it already sized for every row count.
…ials buffer before any launch Fused, clustered and unsplit plans still take zero storage. Host tests run the Nemotron ModelOpt wiring against a recording CUDA boundary: the refusal at 1, 15 and 255 rows in every weight mode, released launch parameters for the valid calls, the loader and each bound forward and segment on their own scratch, and the real arena holding every planned partial through the fuse boundary. Engine.loadOps and segments.segmentOps are the constructors production calls, so the tests read the same ones. `zig build test-modelopt-host` runs them; `compile-modelopt-startup` compiles native startup into an object without running it.
Keep the original path through sixteen rows. Wider GB10 target calls use two m16 row tiles sharing each loaded weight panel, with the original group FMAs and K-slice addition order. Five pipeline stages keep declared shared memory below the narrow tile's budget. Register separate FP64 and original one-row bit comparisons, plus controls that omit the second row tile. Keep compiler resources pending and retain the existing numerical and performance gates for device qualification. Rebase onto shipped 1.1.0, T145 and the corrected startup and first-token increments. Keep both independent host test groups and append the affine verify records without the off-track MiMo collector. Numerical, compiler-resource and GB10 performance gates remain required.
Use the reviewed changelog draft with only completed increments present in this prefix. Keep independent held work out of the release section. Include the source-build version repair paired with the following package metadata increment. Include the runtime-library and GB10 core-pinning additions now carried in this complete export.
Set the package manifest to 1.1.1. Compare its parsed version with the newest semantic release heading in the actual changelog through a shader-free host test included in test-cuda-host. Keep explicit release overrides unchanged. The paired preceding changelog increment documents the source-build 1.1.0 version-string defect.
Gate private routes by default, limit public exceptions to GET health and fixed assets, and reject key files writable by other users. Cover the dispatcher and successful System One score requests. HEAD authentication refusals preserve representation headers and send no body.
Check the resolved listener, Host and Origin before authentication, interim replies and dispatch. Validate header structure and request targets, make public health read-only, and expose exact host and origin exceptions. Cover HEAD refusal framing on the raw connection and document outer header whitespace.
Carry the reviewed GB10 taskset and container guidance with the measured version scope unchanged. Thanks to @Bizuayeu for the ashhart#1033 measurements.
Keep actionable CUDA vision and rank diagnostics, share RPC defaults between discovery and loading, preserve required-export identity and silence successful fallback. Compile the native HIP server without kernels. Thanks to @ss-cong for ashhart#1030. Retain every found discovery handle through process exit, preserve native refusal rendering without duplicate warnings, and name the platform loader environment. Cover process, TLS and shared-library exit callbacks.
Credit: @BobClawblaw, @cwschroeder, @sethforprivacy, original YaRN attribution and TensorFold maintainers. Follow-up to ashhart#986 and ashhart#1005. Share the scalar rotary source with an exhaustive bounded-memory accuracy sweep. Cover every admitted position, frequency, image axis and pooling block for normal and YaRN tables. Record FP64 absolute/ULP extrema and actual normalization fixtures. Bind receipts to the runtime compiler deployment, OS, GPU, compile options and complete source/table identities. Unrecorded or incomplete deployments refuse numerical probes. Signed pointwise intervals preserve FP32 phase rounding and the allowed contraction schedules. All five changed production launches remain pending. Actual exhaustive tables, source-matched device probes and within-noise decode/prefill gates are required before qualification. Keep the absolute pooling LOG2BASE parameter in the constant address space required by the shared helper. All native and probe bindings remain identical. Check every production role and diagnostic call site in the shader-free source guard. Metal compilation and complete fresh GPU receipts remain required. Co-authored-by: BobClawblaw <288991054+BobClawblaw@users.noreply.github.com> Co-authored-by: Christian Schröder <1188270+cwschroeder@users.noreply.github.com> Co-authored-by: Seth For Privacy <40500387+sethforprivacy@users.noreply.github.com>
Default the share to zero columns while retaining the tensor-unit chip guard and explicit TF_ANE_COLUMNS override. Report loaded columns, fp16 program-weight bytes and exact-output qualification. The fp16 path can change replies, so it does not meet the default no-quality-tradeoffs rule.
Credit the precise rotary contributors and preserve the YaRN attribution. Record Bizuayeu's whole-checkpoint two-rank run on shipped 1.1.0, with its scored-prompt and rank-count limits (ashhart#1040).
Require TF_EXPERIMENTAL=deepseek while whole-model gate ashhart#827 is pending. Check both checkpoint spellings at the native boundary, the dedicated server and the whole-model factory before device or payload work. Keep the existing family implementation and shared infrastructure, and label the release notes and known issue accordingly.
Document equal grouping for lone and batched prompt rows, the possible reply change below 128 rows at the same precision, and TF_GLM_PROMPT_K=shape for the previous grouping. No runtime change.
Scope optional dependency warnings at the follower boundary and restore the previous state on every return. Extend the host warning-count control and check the actual follower entry wiring.
Record patch 67 qmm_group and qlinear postimages and patch 71 cuBLASLt and NCCL diagnostic postimages, retaining the original manifest and existing device holds. Align the cluster fake-node labels with the generic names its assertions already expect. No production or kernel changes.
Scope benchmark-address admission to the configured named HTTPS Hub origin while retaining certificate verification and pinned DNS. Snapshot HF_ENDPOINT before child filtering and preserve policy, transport, offline and HTTP failures through IPC and model recommendations. Add app-runner controls for CIDRs, origin policy, endpoint builders, child environment and error copy. Node execution and typecheck remain for the app gate. Thanks to @samwang0041-star for reporting ashhart#1037.
Do not classify an isolated drive fragment in instruction bytes as a private path. Keep full payload and compressed-image inspection, requiring another separator and named component after a drive prefix. Preserve planted source, build-root, UTF-16 and malformed-image controls.
Use @easthambuilds in release and benchmark credits, preserving commit authors and email identities.
Restore @ZackSample's fn_copy_cols kernel from ashhart#896 (e2779f6), which the Living Weights restore omitted while keeping its eager lookup. Keep the original serving prefix and record the additive export hash. Check literal device function names against CUDA and HIP source tokens in the plain test set, with macro and C++ namespace controls. Co-authored-by: ZackSample <134018356+ZackSample@users.noreply.github.com>
Restore the shipped 16-row target prompt MoE cutoff after verification capacity grew to 64. Prompt pieces above 16 retain their bf16 expert recipe; verification continues to use the decode experts. Exercise the real selector at 16/17/32/64/65 rows and every prompt piece through 128, with verify capacity still 64. GDN, shapes, MTP and argmax are unchanged. The model token-identity gate remains separate.
The release CUDA registry has no DeepSeek backend. Refuse both model type names before CUDA work regardless of TF_EXPERIMENTAL, without changing the separate development server's opt-in. Correct the current release notes and known issues while preserving the implementation list, sm_90 floor and credits. Host controls exercise the actual admission function with controlled environment values.
Keep whole-prompt batching available while stepped serving returns fresh requests to the filling queue. Exercise the production capabilities through the real worker and clarify the equal-precision prompt reduction policy from ashhart#1014.
ツリーは upstream の v1.1.1 タグと同一にした。 フォーク独自の EXL3 ポート（zig/src/families/glm/cuda/ 配下と関連カーネル、テスト、ノート）と、 ポートのために共有ファイルへ入れた変更は捨てる。 3bit の EXL3 エキスパートは次のコミットで upstream のエンジンに足す。 Refs #2 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
upstream v1.1.1 の GLM CUDA エンジンは 4bit の EXL3 エキスパートだけを受け入れていた。 共通の grouped カーネル（tf_exl3_grouped_1）は K2=2〜16 を実行時に扱えるので、 GLM のローダー側で次を変えて 3bit（K2=6）を通す。 - admit: quantization_config の bits が 3 か 4（整数、または整数値の浮動小数）なら受け入れる - load: trellis の幅を形状から読み、タイルのバイト数と k2 をその幅で決める。層内で幅が揃わなければ拒否する - 融合 prompt 経路は 4bit 専用のまま。3bit の層は prompt でも grouped 経路を通り、scratch もその大きさで確保する Refs #2 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CI の lean チェック（comment past 120 columns）が新規違反として検出したため。 Refs #2 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
upstream の Windows ビルドは 1.1.0 以降壊れており、upstream 自身の no-gpu の実行でも zig-windows はほぼ毎回失敗している。このフォークでは upstream が直すまで zig-windows を continue-on-error にして参考扱いにする。 あわせて actions/checkout と mlugg/setup-zig を pinact で commit SHA に固定した。 Refs #2 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ジョブ単位の continue-on-error ではワークフローの結果だけが成功になり、 PR のチェックでは zig-windows が失敗のまま表示されていた。 3 つのステップに付け替え、ジョブは成功、各ステップの失敗は注記として残す。 upstream の Windows 修正（ashhart#1034、1.1.2 の系列に入る予定）を 取り込んで zig-windows が通るようになったら外す。 Refs #2 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 19, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
upstream（ashhart/TensorFold）の v1.1.1 を取り込みます。
GLM-5.3-Flash の CUDA 実行は、フォーク独自の EXL3 ポートをやめて upstream のエンジンに切り替えます。
フォークのポートだけが持っていた 3bit の EXL3 エキスパートは、upstream のエンジンに足して残します。
Closes #2
変更内容
このブランチの主な変更は、次の 2 コミットです。
ほかに、lean チェックのためのコメント修正と、CI の設定変更（後述）のコミットがあります。
1つ目はマージコミットです。
ツリーを upstream の
v1.1.1タグ（b10231b94f）と完全に同一にしています（git diff v1.1.1 f0e09b0bは空）。フォーク独自の EXL3 ポートは捨てます。
対象は
zig/src/families/glm/cuda/配下の約 45 ファイル、glm_exl3d.cuなどのカーネル、zig/tests/cuda/glm/、tools/glm_lanes_check.pyです。ポートのために共有ファイル（
engine_api.zig、lanes/、server/chunk_plan.zig、qmm.cu、no-gpu.ymlなど）へ入れた変更も同時に消えます。誤ってコミットされていた macOS のメタデータファイル（
._*）も消えます。2つ目のコミットで、upstream の GLM の EXL3 ローダー（
zig/src/families/glm/cuda_exl3.zig）に 3bit を足します。v1.1.1 のローダーは
bits != 4を拒否していました。一方、共通の grouped カーネル（
tf_exl3_grouped_1）は、半ビット幅（K2）が 2〜16 のものを実行時に扱えます。そこで、ローダー側を次のように変えました。
quantization_config.bitsが 3 か 4 なら受け入れます。整数でも、3.0のような整数値の浮動小数でも構いません。dim(2) = 16 * bits）から読み、タイルのバイト数（32 * bits）と k2（2 * bits）をその幅で決めます。1 つの層の中で幅が揃わない場合と、層ごとに幅が違う場合は拒否します。
scratch を 1 つの幅で確保しているためです。
3bit の層は prompt でも grouped 経路を通り、scratch もその大きさで確保します。
判断したこと
d1018b318fも挙げていますが、取り込み元はリリースタグのv1.1.1にしました。b10231b94fからd1018b318fまでの差分は 5 コミット（README、ドキュメントの訂正、デスクトップアプリ beta 8、アプリのテスト）で、エンジンには関係しません。zig/src/families/glm_cuda/、zig/kernels/cuda/glm/）はフォークにも upstream にもありません。フォークのポートのファイルはすべてフォークにしかなく、「upstream 側へ戻す」対象がありません。
そのため、ポートをまるごと upstream のエンジンに置き換えました。
2026b1116eの再適用：この修正はフォークのポートのglm_kernels.zigだけに効くもので、ポートと一緒に不要になりました。TF_GLM_BUSY_ROWS、TF_GLM_MULTI_WINDOW（32 行の共有ウィンドウ）、TF_GLM_COMM（RoCE の RDMA write による all-gather）などです。upstream も共有ラウンドと RoCE の all-gather（cuda: a one-shot RoCE all-gather for two ranks (roce.zig, roce_gather.cu, tp_link.zig) ashhart/TensorFold#852）を別の形で持っていますが、2 台の GB10 で両者のスループットを比べた計測はありません。
正しさは共通カーネルの K2=6 の分岐に依存し、速度はフォークと同等とは限りません。
3bit の prompt は融合経路を通らないので、4bit より遅くなると推測しています。
CI：このフォークでは Windows のチェックを参考扱いにする
no-gpuのzig-windowsでは、3 つのステップ（zig build test、test-golden、fuzz-cortheon）それぞれにcontinue-on-error: trueを付けました。ジョブは成功扱いになり、各ステップの失敗はチェックの注記として残ります。
あわせて、
no-gpu.ymlのactions/checkoutとmlugg/setup-zigを pinact で commit SHA に固定しました。理由は、upstream の Windows ビルドが 1.1.0 以降壊れていることです。
upstream 自身の PR でも、
zig-windowsはほぼ毎回同じコンパイルエラーで失敗しています。エラーは今回変更していないファイル（
server/speech.zig、core/digest_key.zig、fabric/tp/など）で出ていて、実行されたテストは全件通っています（1423/1438、15 件はスキップ）。upstream には、Windows ビルドを直す PR（ashhart#1034）があります。
メンテナーは、この PR を 1.1.2 の系列に取り込むとコメントしています。
ただし、この PR の土台は v1.1.1 より 91 コミット古いため、v1.1.1 で追加されたファイルのエラー（約 11 ファイル、18 か所）は残ります。
このフォークの目的は 2 台の DGX Spark（Linux）での運用で、Windows では使いません。
そこで、ここでは移植せず、upstream が直すのを待つことにしました。
設定を外す条件：upstream の Windows 修正を含む版（1.1.2 以降の予定）を取り込み、
zig-windowsがcontinue-on-errorなしで通ったら外します。検証
macOS（Apple Silicon）で zig 0.17.0 を使い、次を確認しました。
zig build test-exl3-prompt-host test-glm-admission-host test-glm-prompt-settings-host：7 件すべて通過しました。exl3_prompt_host.zigには、3bit の層では prompt の scratch が grouped 経路と同じ大きさになることを確かめるアサーションを追加しています。zig build compile-glm-native-open compile-glm-prompt-gate-host：Linux（aarch64）向けのコンパイルに成功しました。cuda_exl3.zigの受け入れ判定テストを直接実行しました。bitsが 3、3.0、4 なら受け入れ、3.5 と 2 は理由を示して拒否することを確認しました。次の 3 点は、このマシンでは確認できていません。
zig build test全体は、このマシンに Metal Toolchain がなく、Metal のビルドで失敗しました（cannot execute tool 'metal' due to missing Metal Toolchain）。変更と関係しない環境の問題です。
代わりに CI の
no-gpuのzigジョブ（Linux）で、zig build test、test-kernel-contract、test-golden、fuzz-cortheon、lean チェックがすべて通ったことを確認しました（run 39287933183）。zig/tests/cuda/glm_exl3_tests.zig）は、このマシンでは実行できません。GPU テストは 4bit だけを扱っていて、3bit の GPU テストはありません。
TF_GLM_PREFILL_REDUCE=splitの起動時プローブ、DFlash2 を付けた返答が"draft": falseと一致すること）はまだ行っていません。3bit のチェックポイントでも同じ確認が必要です。
NVFP4 は upstream のリリースノートどおり既知の問題のままです。
🤖 Generated with Claude Code