Landing next #286

TensorFold 1.1.1 is the current release. Python serving remains frozen at v0.6.6; new engine work targets Zig on Metal, CUDA and the opt-in backends.

We plan a patch release every day or two. A PR that fails its gates moves to the next one; a listed dependency or planned feature is not a release date or a claim that its checks have passed.

Shipped in 1.1.0

1.1.0 shipped the Qwen CUDA base and shared EXL3 operators, Flash Next CUDA, Gemma 4, Bolt/RADIO video, and Nemotron cache and compatibility fixes. Drama 3 provides native speech with the checked default-voice HTTP subset, credited to @plotarmordev and @drowzeys.

Flash Next on Metal defaults to the single host; explicit parallel counts above one select shared sessions. The separate shared-session cache-count and lone-decode performance repairs remain deferred. Windows remains from source. H3/FastH3 and Qwen-Image ship experimental tools/library components; text encoding and final media processing still use external pipeline pieces.

The GLM-5.3-Flash CUDA pair shipped through #976 with #855, #984, #995, #979 and #1007, including EXL3, media, index rings, packed linears, DFlash2 and kept-state repairs. Thanks to @BobClawblaw and @heitke. A community run on two GB10s at TP=2 now loads and serves the whole ModelOpt NVFP4 checkpoint on 1.1.0, reported in #1040 by @Bizuayeu; scored prompts and more than two ranks remain untested.

Shipped in 1.1.1

• Qwen CUDA's larger verification trees, shared stream rounds, batched DFlash proposals and measured per-stream width choices, Qwen3.5/3.8 on CUDA: decode 37-73% faster on #836's engine, bits unchanged (on #836) #978. Startup window costs are cached between processes; cache hits still warm live resources, and short-context DFlash2 sessions no longer need room for the calibration windows. The 27B Metal rollback bank allocates lazily where partial state is needed; a new physical-memory peak comparison remains unmeasured. Thanks to @BobClawblaw and the existing Qwen contributors.
• Flash Next Metal image/video support, media-aware prompt-cache identity and the 1,048,576-token window, Flash Next: images and video on native Metal, and warm passes that keep nothing skipped (stacked on #606) #986 and Flash Next on Metal: a 1,048,576-token window with static YaRN x4 (stacked on #986) #1005. Windows beyond the trained range use static YaRN on one Mac; paired long-window loads are refused. Rotary preparation and index pooling use precise math, including M-RoPE. The separate shared-session repairs above are not included. Thanks to @BobClawblaw, @cwschroeder and @sethforprivacy, retaining the original YaRN credit.
• Shared CUDA graph-cache/arena, tensor-parallel transport, session-storage and lane facilities, cuda: a keyed graph cache and a device arena, off until a family installs them; failed launches name their kernel (DeepSeek-V4.1 stack 2/14) #635, fabric: tensor parallelism across CUDA processes (bootstrap, NCCL, mailbox and RoCE collectives, plan link, fail-fast) (DeepSeek-V4.1 stack 3/14) #636, sessions: session memory for paged-KV families (page pool, prefix trie, RAM residency budget, streamed NVMe tier) (DeepSeek-V4.1 stack 4/14) #637, server: graceful drain and pre-token keepalives (DeepSeek-V4.1 stack 7/14) #663 and CUDA and sessions: runtime APIs, graph snapshots and checked history for DeepSeek-V4.1 #682, used by participating families. The server drains active requests on shutdown and sends pre-token streaming keepalives. Thanks to @jayleaton and the original runtime and lane contributors.
• DeepSeek-V4.1 CUDA implementation pieces, DeepSeek-V4.1: host configuration, EXL3 packs and rank weight plans #667, DeepSeek-V4.1: Engram shard rows, cache and window read-ahead #668, DeepSeek-V4.1: CUDA sources, launch bindings and compiler integration #671, DeepSeek-V4.1: device weight loading and decode block emitter #674, DeepSeek-V4.1: KV memory, session admission and round planning #686, DeepSeek-V4.1: forward execution, decode graphs and device KV exchange #688, DeepSeek-V4.1: prefill segments, CED and long prompts #691, DeepSeek-V4.1 L11a: keyed sampling #693, DeepSeek-V4.1 L11b: host drafting and lanes backend #694, DeepSeek-V4.1 L11c: DSpark device modules and slot drafting checks #695, DeepSeek-V4.1 L11d: batched rows, slots and round execution #699, DeepSeek-V4.1: serving host, release tokenizer and the server's family hook #702, DeepSeek-V4.1: served engine, model boot and CLI #703, DeepSeek-V4.1: performance hooks and host checks #707 and DeepSeek-V4.1: performance kernels, family images and device bench runners #727, with shared contracts from DeepSeek V4.1: the Engram token map built in Zig, byte-identical to the release's #824, DeepSeek V4.1: the family — configuration, Engram tables, resident weights and the Metal operators #825, DeepSeek V4.1: fp64 reference tests for the attention and hyperconnection operators #826 and DeepSeek V4.1: the lane backend — one stream per slot on the lane core #827. No release binary serves DeepSeek checkpoints until the whole-model gate in DeepSeek V4.1: the lane backend — one stream per slot on the lane core #827 passes; release binaries refuse them and have no environment opt-in. A complete Metal backend remains separate. Thanks to @jayleaton and @akol1, with common EXL3 credit to @justincasey, @BHCC2025, @vcruz305, Turboderp and existing contributors.
• Kolibri-1 text research-checkpoint serving on native CUDA, kolibri1: Kolibri-1 on CUDA through the lane host, drafts off (#481) #991, through the common scheduler, prompt cache, shared rounds and copy drafts. EAGLE-3, a neural drafter and vision remain unsupported. Thanks to @jschmied, @jayleaton and the common operator contributors.
• GLM CUDA stepped serving interleaves fresh prompt chunks with running decode. Whole-prompt serving can batch eligible fresh text prompts across ranks, keeping each sequence's final prompt row and first draft separate, with at most 16 prompts and 2,048 rows per pass (GLM-5.3-Flash on two CUDA ranks: prompts admitted together (prefill_many) #1014). Prompt log probabilities, structured output and required/named tool calls are included (GLM-5.3-Flash on CUDA: prompt_logprobs on /v1/completions #1016, GLM-5.3-Flash on CUDA: structured output (first_masked and held) #1017, Required tool calls in GLM's markup: xgrammar's glm_xml style #1018); image, resumed and scoring paths keep their appropriate policies. Prompt projections use one K slice at every operator row count by default. This equal-precision regrouping can change replies versus 1.1.0, including long or resumed prompts with internal projection calls under 128 rows. TF_GLM_PROMPT_K=shape, with TF_GLM_MM=1 on every rank, selects K slices by shape throughout prompt projections, including full blocks; it does not restore 1.1.0's mixed row-count policy. Verify and draft windows keep their existing policy. Thanks to @BobClawblaw and @Bizuayeu.
• The later cuda/tree_round: core/tree_round's verify and commit step on CUDA, shared by every family #734/cuda: SwiGLU, output gates, Gated DeltaNet inputs, QK norm and RoPE for every family, checked against float64 #797 CUDA contracts and the aot: a launch's 1 may take the runtime-parameter variant after every folded candidate #1020/aot: a launch with a runtime argument of 1 fails against a capture that lacks the value-1 constexpr variant (the unspecialized cubin computes the same bits) #1003 launch-fallback and capture work. Failed launches include their dimensions; fallback retains the preferred matching order and capture checks. These are the later contributions, separate from the heads already in 1.1.0. Thanks to @jschmied, @GustavBlack and @cwschroeder.
• Benchmark v2 stop/delivery accounting and explicit chat-only routing, tensorfold benchmark: one sampled reply that stops before 256 tokens makes the cell incomplete (Nemotron oQ8, code T=1) #969 and tensorfold benchmark: the code workload's raw /v1/completions prompt gets empty output from DeepSeek-V4.1 at T=0 #1019. V1 stays unchanged; both v2 publication paths remain disabled pending receiver support and the separate ranking rollout. Thanks to @BlivionIaG, @BobClawblaw, @easthambuilds and @jayleaton.
• Request-owned completion times and tok/s, Per-request times on the done line #1025, and fresh sampled-request seeds with explicit replay and configured-default precedence, A request without a seed always gets the same sample #1026. Thanks to @tpischke and the existing server contributors.
• FP4-Local host foundations, core: decode and encode csf_32x2_v1 block scales on the host #1021, native/cuda_memory: count every file the weight map names in weightBytes #1022, core/modelopt: read MIXED_PRECISION's per-module recipes #1023 and cuda/fp8: pack a rank's share of block-FP8 scales from inside a 128 block #1024: block-CSF scales, indexed weight-size refusals, per-module mixed-precision recipes and rank-local block-FP8 scale layout. These are not a complete FP4-Local family loader; mixed-format consumers, stored MXFP8 MTP, F32 norms and GPU/model qualification remain separate. Thanks to @Bizuayeu, @jschmied and local-inference-lab for the format/reference work.
• Native CUDA and recording wrappers preserve the family's first-token handoff flag, fix(cuda): preserve first-token handoff through native host #1031; contributor links follow the same account's handle change, docs: update contributor links to jrb-ai #1036. Thanks to @jrb-ai. Missing optional-library diagnostics name the library and feature, with dependencies in capabilities --json, 1.1.0 native CUDA (aarch64, 2x GB10): GLM-5.3-Flash Mia EXL3 checkpoint fails to load with LibraryUnavailable #1030; thanks to @ss-cong. The runbook documents GB10 efficiency-core pinning, GLM on CUDA, GB10: the native engine spins two CPU cores per rank, and long runs of replies overheat the CPU #1033; thanks to @Bizuayeu. The engine's idle-spinning fix remains separate, so GLM on CUDA, GB10: the native engine spins two CPU cores per rank, and long runs of replies overheat the CPU #1033 stays open.
• Living Weights implementation and safety work, Living Weights: a quiet refit holds an earlier fact's neuron on its own questions as read now #876, Living Weights: every earlier fact asked again after a lesson, and a take-out that puts the pocket back bit for bit #877, Living Weights as a core module: the learner, pocket and fits in zig/src/living, a family described as data (Nemotron only) #878, Living Weights: a later lesson's quiet refit can silence an earlier fact on its own questions, and nothing asks it again #870, Living Weights: facts learned and checked in conversation, not only asked cold #891, CUDA input gradients for Living Weights on Flash Next: a Gated DeltaNet layer, 4-bit products, routed experts, gated attention (shared), and the hyper-connections and pocket (family) #895, Living Weights on Qwen3.8 Flash Next (CUDA, GB10): the pocket in every pass, the trainer, and --slide #896, Living Weights on Qwen3.8-27B (CUDA, GB10): the pocket in every pass, the trainer, --slide, and pocket neurons that no longer drive one another #911, slide: /v1/slide/learn takes questions beside its text #992, Proposal: Living Weights: accept caller-supplied questions and answers #687 and Docs: adding Living Weights to a family #905, is included in the tree. Its model, recall, retention, take-out, training and save/restart qualification remains pending. The Sliding Weights lesson server: rejected lessons do not shift later training checks #1029 lesson-counter repair is excluded and remains held. Thanks to @ZackSample, @philip-pentatonic, @akol1 and @BobClawblaw.

1.1.1 also requires a valid Host, checks loopback names and browser origins before routing, and protects routes with configured keys while retaining the documented public exceptions. Key files writable by group or others are refused. Qwen3.8-27B Metal keeps Neural Engine prompt columns off by default because fp16 prompt rows can change replies; TF_ANE_COLUMNS=N is an explicit opt-in.

The Hub fake-IP and validated-endpoint correction, #1037, is integrated and ships in the next app beta. Private and metadata addresses remain refused on the Hub path. Thanks to @samwang0041-star for the report.

Qwen3.8-27B cold prefill at 64K on sm_120 is about 2% slower than 1.1.0 in three reported comparisons. Follow-up work is queued for 1.1.2; this is not a claim that the regression is fixed.

Landing next: 1.1.2

These are planned entries, conditional on their final integrated gates. Ready means ready to carry into the candidate, not released or fully qualified. We will do the rebases and retain contributor credit.

Flash Next Metal prompt steps for #1032, bounded Metal residency with real idle keepalive work, CUDA per-stream drafting precision and the server's total read budgets and connection cap are prepared for review and integrated checks. #1032 stays open until the prompt-stepping fix ships. The Flash Next Metal startup-admission work for #1039 still needs complete image-memory accounting, failed-load cleanup and known-fitting controls before it is ready. We are also investigating the sm_120 64K regression above; no speed recovery is promised yet.

#1048's Gemma 4 CUDA proposal is accepted, thanks to @jschmied for asking first. Start with the shared CUDA operators; the image path is part of the support bar. Shared attention and optional rounding controls must preserve current callers. This accepts the work, not a completed backend or a guaranteed 1.1.2 landing.

Held for a later 1.1.x

MiMo-V2.6-Flash, #728 and issue #725, is not included in 1.1.1. Its current CUDA path needs a Python-captured Triton kernel set that TensorFold does not ship. Native kernels for its eight Triton launches must come first, then full-model fidelity, draft/plain, resumed/fresh, concurrent/solo, two-rank failure and compiler/resource gates. Thanks to @ajensenwaud, @jayleaton and the common EXL3 contributors.

Living Weights qualification and #1029 remain held as described above. DeepSeek serving waits on #827; there is no release opt-in. Embeddings, mixed Metal rounds, GLM context-parallel prompt work and opt-in FP8 KV remain separate proposals or profiling work, with their own approvals and gates; none is a promised 1.1.2 feature. The separate Flash Next shared-session repairs remain deferred.

#606 is integrated in 1.1.0; #874 is parked pending its author's resumption, and #930 needs the requested corrections. We will keep a clear outcome on every thread: a merge, integration with credit, an explained closure, or an author-agreed pause. #593 and #832 are covered by their released integrations; #847 stays open until DeepSeek's serving gate passes. #840 and this landing tracker stay open.

We merge contributor commits where they fit cleanly and retain their authorship. When integration needs a rewrite, we credit the original contributors by @handle and distinguish the maintainer changes. We will do the rebases for authors and carry small agreed fixes at landing. Please provide the pinned source and checkpoint revisions, reproducible commands, public fixtures, full token hashes, reference comparisons, memory measurements and before/after performance receipts, with failures and skips stated plainly. A changed head needs its affected checks refreshed.

Every integrated candidate must pass host builds on Mac and Linux, the host suites and kernel contract, then drafted/plain, resumed/fresh and concurrent/solo checks for every family and backend it touches. Changed arithmetic needs independent FP64 bounds and teacher-forced fidelity checks. Native image paths need their own processor, tower, embedding splice and cache identity checks; an exceptional temporary refusal must be named explicitly. Shared lanes and drafts stay in the engine, and NVIDIA's checkpoint math stays the default. Privacy, secret and copied-code checks include controls that prove the scanners detect planted findings. Release testing also covers cold prompts, long contexts, memory classes, archives and installation, against 1.1.1, 1.1.0 and the applicable reference server. Only work that passes its gates ships.

The contribution rules and receipt format are in CONTRIBUTING.md; the released capabilities and remaining qualifications are in the changelog.