Repository navigation
Conversation
…alls them. graph_cache.Cache captures one executable graph per caller-named key and replays it when the key comes back; ranks agree on every capture, eviction and fallback through an Agree hook, and a memory floor turns misses eager. arena.Arena sub-allocates DeviceBuffer.alloc from a few large chunks (VMM or plain cuMemAlloc) through memory.setHook; borrowed spans and pointers it does not own are freed as before.
…; CUresult names fall back to a table when the driver cannot answer. cuFuncGetName is optional (CUDA 12.3+), so an older driver still opens. During the exit's teardown cuGetErrorName fails too, which printed CUDA_ERROR_DEINITIALIZED as "CUDA_ERROR_UNKNOWN_CODE (4)"; code 4 now says the process is exiting.
Use CUDA device attribute 110 for GPUDirect RDMA with VMM support. Attribute 126 reports the memory sync domain count and can incorrectly enable the RDMA allocation flag. Add a regression for the external ABI contract.
…r stays under the max, a max of 0 is eager - A capture whose agreement throws, or whose insert runs out of memory, no longer leaks its executable: the map's slot is reserved before the capture, and the executable is owned here until the insert, which cannot fail. - A host map that cannot grow counts as no room in the existing room vote, so every rank declines that miss together. - `min_keep` is clamped to `max`, so under the memory floor the cap never rises above the configured max. A max of 0 runs every step eagerly, with no collective. - A cap lowered to 0 under the floor goes back to max once every rank has room, instead of staying eager for good. - Hits make no collective: with `Agree` set, keys, settings and fingerprint changes must already match on every rank. The docs now say so: the fingerprint changes at the same step on every rank, and `fingerprintOf` is for one rank. Co-authored-by: Jonathan Regan-Beasley <197533392+jregan-beasley-smc@users.noreply.github.com>
The arena's `live` (small ranges) and `large` (own chunks) maps drop an entry on every free and gain one on every allocation for the life of the process. std's tombstones never clear, so in time they take every free slot, and each allocation, free and miss then probes the whole map. Each map now counts its removals and is rehashed in place once they reach a tenth of its capacity. Test: 20,000 random allocations and frees (small ranges and large chunks) keep a tenth of each map free at every step; without the rehash it fails.
…d RoCE collectives, plan link, fail-fast. zig/src/fabric/tp brings a rank up from TF_TP_WORLD, TF_TP_RANK, TF_TP_MASTER and TF_TP_PORT, agrees the settings digest over every rank, and gives a family all-gather, all-reduce and broadcast over NCCL (TF_COMM_BACKEND=nccl, the default), a pinned host mailbox for ranks on one host, or RoCE verbs between two hosts (TF_TP_ROCE_*), with an NCCL fallback when RoCE setup fails. Rank 0 sends each round's plan to the others over its own link (TF_TP_PLAN_*). A failure or a vanished rank ends every rank shortly after (TF_TP_FAILFAST, TF_TP_FAILFAST_GRACE_S). No family uses it yet. zig build test-tp runs the host tests; zig build tf-tp-test builds the two-GPU runner (-Dtp-nvcc= or -Dtp-fatbin= for the mailbox kernel). The RoCE all-gather adapts b12x's RoCEnante protocol.
… the stop word with no fail-fast exit; a rank killed mid-drain still fails the other fast.
… /24 the peer also has The automatic pick took the first N active RoCE ports in device-list order and never checked that the peer can reach them. On a host with two neighbours, one picked port can sit on a subnet the peer does not have, and the queue pair's RTR step then fails, which reports RoCE unavailable. Every rank now sends its usable ports' IPv4 addresses over the bootstrap. Each rank then keeps the ports whose /24 the peer also lists, ordered by subnet and each /24 no more often than both sides have it, so both ranks pick the same number. With no shared /24 the old first-N order is kept, with a warning to set TF_TP_ROCE_HCA. The exchange runs on every rank, including ranks that set TF_TP_ROCE_HCA, so the bootstrap stays in step.
…FNAME and TF_TP_MASTER are needed, with a two-host example
… kernel backends up front; run_hosts.sh passes HCA picks - `tf-tp-test kernel` reports the embedded kernel and exits 1 without one. `suite` and `failfast` on the mailbox or RoCE backend stop at once with a message that names `-Dtp-nvcc`, instead of failing later on a missing kernel. - `run_hosts.sh` stops before any step when the kernel is missing. It exports TF_TP_ROCE_HCA and NCCL_IB_HCA (derived from TF_TP_ROCE_HCA when unset), warns when a host has more than one RDMA device and neither is set, and records all three variables in host.txt.
… with another region layout are refused roce_guard.stripes computes each HCA's share of a post (its send stripe, the peer's receive range and its flag word) and refuses with error.OutOfRegion any range past its slot, its flag area or the registered region, before post_send. The verbs wire and the shm wire both post through it. The shm wire moves to roce_shm.zig and can stripe like the verbs wire (Options.shm_stripes), with an optional tap that sees every post for tests. The connection record carries the slot and registered length, so a peer with another layout is refused at connect; its magic is bumped. Tests: stripes tile the slot over two HCAs and refuse a post past the slot, a third slot, an unpadded or empty post, no HCA and a short region. On fake verbs a post past its slot reaches no verb, and a full slot stays inside the region. Over the shm wire striped as two HCAs, 2,000 variable-length gathers stay inside slot and region, bit for bit.
…would ask for one A post of about 9.6 MB faulted a DGX Spark's HCA. roce_guard.max_write (1 MiB, a constant) is checked in stripes() before the slot bounds, so the verbs wire, the shm wire and the proxy refuse a larger post with error.WriteTooLarge before any verb, whatever the slot size. Roce.init refuses a slot past it. Config.parse fails boot with a clear message when TF_TP_MAILBOX_MAX_KB would make the RoCE slot pass 1 MiB; the value is never clamped. Tests: a 9.57 MB and a 1 MiB + 16 post are refused in a 10 MiB slot and reach no verb, a 1 MiB post goes out; boot refuses TF_TP_MAILBOX_MAX_KB 1028, 2048 and 16384 and keeps 1024.
…sized ops, at most 2 MiB in flight Default off. allGatherV strides past the slot, up to TF_TP_ROCE_CHUNKED_KB, go RoCE as ceil(n / slot) launches of the tp_roce kernel: op c moves at most a slot of each shard from c x slot (RoceArgs.chunk), and an op empty on both ranks rings no doorbell and leaves the epoch. Every RDMA write keeps the 1 MiB cap. The kernel poisons the transport rather than stage past a slot, and its four failure-record blocks fold into fail_record (same stores and fences). VerbsWire.post waits for completions while the bytes posted and not completed over both HCAs would pass max_inflight (2 MiB, a constant in the post path); a wait that sees no completion for 10 s fails the proxy. Boot refuses TF_TP_ROCE_CHUNKED_KB when the kernel image lacks the tp_roce_chunk_abi marker. The setting is part of the digest both ranks agree on. Tests: chunked strides from 300 KB to a 128 MiB union posted back to back on fake verbs whose completions lag keep at most 2 MiB in flight and no write past 1 MiB, and reach the cap; the wire's own in-flight count and peak match the fake HCAs'. Over the shm wire, 2,000 chunked variable-length gathers of up to four slots arrive bit for bit.
…region; the region is DONTFORK and NOHUGEPAGE A local protection error can arrive while the IOMMU invalidates its mappings lazily. On such an error the old path left the other HCA's QP ready to send (still taking the peer's writes), hung up the fate channel without telling the peer to stop, and freed the region at once: unregistered from CUDA with no GPU sync, then QP destroy, MR deregistration and munmap. A device write after the unmap could then reach a reused page through a stale IOTLB entry. - Roce.fail and Roce.abort (the fail-fast hook) move every QP to the error state: no send goes out and no peer write lands. - Roce.deinit quiesces first: QPs to error, every posted request reaped (flushed or not), cuCtxSynchronize. A failed transport, or one that does not quiesce, keeps its region registered and mapped; the process exit tears it down. - Session.close on a failed transport goes fail-fast before closing, so the peer aborts and halts its HCAs first. - The region is MADV_DONTFORK and MADV_NOHUGEPAGE before ibv_reg_mr; a refusal fails the start. - A QP whose actual max_inline_data is under 4 B is refused (the flag post uses lkey 0 on a stack word). - One log line a rank: region base and length, each HCA's lkey and rkey, the peer's region and rkeys. Tests: on fake verbs Roce.abort halts both QPs; quiesce halts both before the first flush and reaps every posted write; an aborted transport reports itself failed, and over the shm wire the aborted rank keeps its region while the other frees it; smaps shows dc and nh on a prepared region; the inline bound refuses 3 B and takes 4.
…path on real HCAs, only on an idle link - `tf-tp-test failfast V transport [N [HOLD_S]]`: N clean 4 KiB sums, the ranks meet over NCCL, each waits for its link to go idle (every HCA's posts completed), both agree over NCCL, then the victim takes the proxy's failure path (`Roce.inject`, which is `Roce.fail`) and closes its session: both ranks halt their QPs and exit 70, and the victim's region stays registered. When either rank's link is not idle there is no inject, a clean stop, REFUSED and rc 2. - Its host-only forms (TF_TP_FAILFAST=0, or a hold) are refused unless TF_TP_ALLOW_HOST_ONLY=1 and no RDMA wire is in use (the shm wire, or no RoCE), so they never run on an HCA. - `run_transport.sh MASTER PEER_SSH OUT`, driven from rank 0's host (rank 1 over ssh): prechecks on both hosts (no GPU process, memlock unlimited), then T0 (suite), T1 and T3 (each rank as the victim). After each step it reads both hosts' kernel log since the step and every HCA *err* counter; a hit, a counter that grew, an unexpected rc or a failure line prints STOP: and exits 1. `run_transport.sh t2 OUT` runs the host-only form on one host. - The `halted` and `quiesced` lines carry each QP's state read back with ibv_query_qp (an optional symbol), each HCA's posts and completions, the completions the drain reaped and how many of them were flushes. - `Roce.allGatherV` refuses a stride past the slot (one op; the route never sends one). - The README states the RoCE limits and how to run the check. Tests: on fake verbs the link reads busy with two posts out and idle after the drain, and the report line names both QPs in the error state with every completion a flush.
The prefix index keeps one node per distinct full page of every saved entry in `by_digest`. Each saved turn adds its new pages, and each entry dropped or evicted removes the pages no other entry passes through, for the life of the server. std's tombstones never clear, so in time they take every free slot, and each save's new pages and each find's walk past the cached prefix then probe the whole map. The map now counts its removals and is rehashed in place once they reach a tenth of its capacity. Test: 3,000 turns of up to 60 fresh pages, the oldest dropped past eight, keep a tenth of the map free after every turn; without the rehash it fails.
A family can now hand the lane host a round planner (api.Rounds) that admits requests, splits prompts into pieces and runs them in the same rounds as the other streams' decode rows. The planner is opt-in: the host's own loop, with a backend's prefill_step a chunk a loop between rounds, stays the default, and a host with a planner runs the planner's loop instead (LaneHost.run picks one). The backend gains optional entries for this: begin, piece, pieces (several streams' pieces in one call), finish and join, so a round's finished prompts get their first drafts in one backend call. A planner's prompts open through lanes/open.zig like the whole and the chunked pass (open.openedWith; their first drafts sized by the head's reach under node probabilities), and a prompt cancelled at its final releases its lane as a cancelled chunk does. The lane core also takes a family row policy (lanes/trim.zig) in place of the built-in allocation, proposers that price their own drafts, and a timed switch for backends without round timers. Under a planner the host still steps the learner while no request waits or runs, and a failed piece or piece run fails only its own requests. A planner over a backend without prompt pieces (such as the external drafter wrapper) is refused at start with error.PiecesUnsupported. The round loop lives in lane_rounds.zig, a request's stream setup in lane_stream.zig and the pieces' steps in lanes/pieces.zig, keeping the host and the engine under the line limit.
…oundary, and every request queued, filling or running fails with the reason. Opened.halt (optional) lets a server stop an engine between rounds instead of under a running one; LaneHost.halt is that hook for lane-host families. A request still filling its prompt a chunk a round fails too, with its lane freed, and a prompt pass the halt cancels mid-chunk reports the halt's reason rather than a cancel.
…s waits time out The second prompt was submitted after start(), so a loaded scheduler could let the first finish its pieces before the second arrived: no queued admission, and the test's waits > 0 check failed. Both are queued now before the first round. Each wait gives up after 10 s, and a timed-out wait stops the host thread before the test's frame is gone.
family.zig is an optional vtable a family installs in place of the chat template: prompt rendering, thinking and effort fields, a streaming reply reader that parses its own call markup, and the turn it remembers. Without a family the server runs as before. chat.zig's reply collection moves to chat_generation.zig and openai.zig's run to openai_run.zig (pure moves), so neither file passes 600 lines with the hooks in.
…pt loop ends, and again if it exits with connections open.
New generation requests get 503 with Retry-After (code server_restarting; Anthropic overloaded_error) and /health answers "draining" without asking the engine. Requests in progress get TF_DRAIN_S seconds (default 20; a second signal cuts it short), then the engine's optional halt ends its rounds at a boundary and what is left fails with "server restarting: retry shortly". The engine closes before the server is freed, once.
…in check refuses the request (503) and the drain waits for it. A tests-only seam (routes.counted_hook) delivers SIGTERM in that gap; with the drain check moved before the count the test fails.
…ery TF_SSE_KEEPALIVE_S seconds (default 15).
Queueing and a long prefill send nothing after the head, and clients or proxies
that time out a silent body leave and retry from zero. Chat, completions and
responses streams get an SSE comment (": keepalive"); Anthropic streams get its
ping event. A keepalive that cannot be written cancels the request as a client
that left.
…ain checks are done The listener stays open only while a generation runs. Case 1 streamed a "@short" reply (about 180 ms) and then sent a messages request and a /health request expecting 503; when the reply ended first, the drain finished and those requests found the listener closed. The stub engine's "@Hold" reply now waits for the test's release.
…TP weight plan and its named tensors. zig/src/families/deepseek_v41 starts with the parts that need no GPU: the text config (config.json's text_config, with the per-layer CSA2 topology), EXL3 group geometry and a rank's TP slices, the pack's tensor names and shapes, the pack folder read from its safetensors headers alone, the rank's weight plan, its host images and the weights as named tensors in the reference loader's forms. zig build test-dsv41 runs the host tests on the release config, a synthetic pack and a 2.8 bpw pack's expert widths; TF_DSV41_HEADERS adds real header dumps.
… cached on disk for a fast boot. prepare.zig builds the run-time forms from a rank's named host tensors: linear strips and the routed and shared experts' tables; its test checks the dense lanes layout at every width against a reference (fixtures/lanes-ref.bin). prepared.zig writes and reads built layer images under TF_DSV41_PREPARED (_WRITE builds them, _VERIFY refuses a damaged payload), keyed by the loader sources that build them, so a changed loader never reads an older folder.
…ow's tokens, a rank's heads. engram_host.zig lays out the Engram tables of the config, hashes each window's token n-grams into table rows as the reference hasher does (checked on the release config and tokenizer, fixtures/engram-hash-ref.bin), splits the heads over the TP ranks and builds the synthetic index rows the tests use.
…ed phase (TF_DSV41_PHASES). TF_DSV41_PHASES=1 times each named phase of a round on the host clock, =sync also waits for the stream at each boundary, and TF_DSV41_PHASES_OUT=<file> writes the totals as JSON. Off by default; nothing reports into it yet.
…HCA verbs-error naming) into stack/08r-runtime Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…HCA verbs-error naming) into stack/09a-kv Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…HCA verbs-error naming) into stack/09b-forward Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…HCA verbs-error naming) into stack/10-prefill Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…HCA verbs-error naming) into stack/11a-sampling Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…HCA verbs-error naming) into stack/11b-draft Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…HCA verbs-error naming) into stack/11c-device-drafts Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…HCA verbs-error naming) into stack/11d-batched-rows Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…HCA verbs-error naming) into stack/12a-serving-host Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Conflicts, both sides kept: - zig/src/cuda/memory.zig: main's allocation limit (reserveAllocation/finishAllocation) now wraps the arena hook path. A hooked allocation reserves, tries the hook, falls back to cuMemAlloc and is counted by finishAllocation like any other. - zig/src/cuda/root.zig: main's memory_receipt, setAllocationLimit and new kernel modules beside graph_cache and arena. - RUNBOOK.md: main's GLM heat-band paragraph, then the TF_GRAPH_CACHE paragraph. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…r.check does launched() named the failing kernel for every non-success code, so a launch that main's tests answer with CUDA_ERROR_NOT_READY (qwen_cuda_test, experts3_host_test) reached errorName on a fake driver and crashed. It now returns error.NotReady first, like Driver.check. The ranks test reads the monotonic clock through std.Io, so it also compiles on Windows hosts (zig-windows). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…3-tp-fabric Conflicts, both sides kept: - THIRD_PARTY_NOTICES.md: two b12x RoCEnante entries, each naming its own files. Main's two-rank CUDA RoCE gather (zig/src/cuda/roce.zig, zig/kernels/cuda/roce_gather.cu), then the TP fabric's RoCE path (zig/src/fabric/tp/mailbox.cu, roce.zig, roce_verbs.zig). - build.zig: main's sys and checkpoint_config wiring beside tp_build; tp_build.add runs before both walks, so the TP modules (which compile zig/src/cuda/root.zig, now importing sys) get the sys import. Auto-merge fix: - zig/src/cuda/nccl.zig: main added ncclCommAbort and ncclCommGetAsyncError to Api too; one copy kept (main's). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Conflict, both sides kept: - build.zig: sessions_build.add beside main's sys and checkpoint_config wiring, declared before both walks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Conflicts, both sides kept: - zig/src/server/openai.zig: main's deferred stream opening (Sink.begin, unsent before the stream opens) with the SSE keepalive; chat.generate calls begin before its token loop, so a keepalive only follows an opened stream. - zig/src/server/routes.zig: main's /v1/audio/speech joins the counted dispatch as is_speech, so a draining server refuses it with 503 + Retry-After like the other generation routes. - zig/src/server/status_routes.zig: /health answers "draining" first, without asking the engine, then main's engine health check (503 "unavailable"). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ashhart#861's lane planners are in main (903f601, be757af), so this layer takes main's lane files and drops its copy: engine_api.zig, lane_halt*.zig, lane_host*.zig, lane_review_test.zig, lane_rounds*.zig, lane_stream.zig and lanes/{backend,config,engine,engine_test,fake,lanes,open,pieces,proposer,recall_engine_test,trim,windows}.zig are main's bytes. Each was byte-identical to ashhart#861's head e6df8ad here; DeepSeek-V4.1's host side imports none of them. - build.zig: dsv41_build.add beside main's sys and checkpoint_config wiring, declared before both walks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…POSIX and Linux APIs its host code reads Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
No conflicts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Conflict, both sides kept: - zig/build/cuda.zig: tf-dsv41-test is installed after tf-cuda-test as before, then main's tf-tp-watch-check, Flash Next checks, compile steps, Qwen contract images and config server. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
No conflicts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Conflicts, both sides kept: - zig/src/cuda/memory.zig: Usage carries main's host_peak and this layer's allocs; main's sample() returns the driver allocation count; the cuMemAlloc/cuMemFree count sits in main's reserve/finish allocation path. - zig/src/cuda/aot.zig: the trace's hit() beside main's has(). - zig/src/cuda/stream.zig: initPriority beside main's borrowed(). - zig/src/cuda/root.zig: pack beside main's attention, MoE glue, dense rows and EXL3 modules and tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Conflict, both sides kept: - build.zig: kv_build.add after the DeepSeek-V4.1 host tests, beside main's sys and checkpoint_config wiring, declared before both walks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ly's other host tests Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
No conflicts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
No conflicts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Conflicts, both sides kept: - build.zig: sampling_build.add joins the DeepSeek-V4.1 host tests inside the non-Windows gate (its pipeline runs over the TP fabric's in-process ranks), beside main's sys and checkpoint_config wiring, before both walks. - zig/build/cuda.zig: tf-dsv41-samp is installed after tf-dsv41-test as before, then main's additions. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
No conflicts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rafts No conflicts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…batched-rows No conflicts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…erving-host Conflicts, both sides kept: - zig/src/server/chat.zig: main's engine-unavailable message for a closed engine; Sink.begin opens the stream, then the family's reader or the tool streamer. Generation lives in chat_generation.zig; main's only changes to it were comment rewrites, carried there and to closeCall and cancel. - zig/src/server/openai.zig, openai_run.zig: Run lives in openai_run.zig and takes main's deferred stream opening (StreamSink.begin, pending deltas, unsent before it opens) and cx.status() for refusals. - zig/src/server/server.zig: Config.family beside main's environ. - zig/build/server.zig: addTo keeps the tokenizer and lanes parameters and gives the server module main's `lanes` import from them; every caller passes the engine API's lanes. build.zig, cuda.zig take main's new server test steps and required-tools imports with that addTo. - build.zig: test-dsv41-serve (portable) after the non-Windows DeepSeek-V4.1 host tests, before both walks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…gh sys.getenv, which builds on Windows Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this changes
DeepSeek-V4.1 serving side: the release tokenizer, chat encoding, DSML tool calls, and the server's family hook that serves them over HTTP. Depends on #699. This cumulative PR targets
main; its net change over #699 (stack/11d-batched-rows) is 42 files.#861: now in main via
903f601c/be757af6; this PR no longer carries the shared lane planners.The server's family render / reader / commit hook lands here (moved from #663 per review). A checkpoint whose
config.jsonhasmodel_type: deepseek_v41serves through it; every other family stays on the chat-template path unchanged.Built on current
main(Recall,/v1/imprint, System One, vision input,Stalerefusals,token_sha256,reply_text.Shownare main's). Changes beyond the originalf5db10bf:prompt.zig: a family render now also gets the prompt-cache rewind (rewind_len), which the template path already had. A family's own prepared images travel asfamily_images, apart from main'simages: []api.Image.deepseek.zig:DeepSeek.loadwrites the missing-tokenizer.jsondiagnostic into the caller's problem arena instead of the general allocator. This fixes the allocation leak the previous description listed as unresolved.main.zig:--recallwith a DeepSeek-V4.1 checkpoint is refused at startup. Recall keys its record on the chat template's tokenizer, which this family does not load.anthropic.zig: a streamed tool value is copied before the reply text under it grows. This restores the hook commit's original line.zig/build/server.zigaddTotakes the tokenizer and lanes fordsv41_serve, andcortheon_fuzz,hipandxpupass them.test-cuda-hostkeeps main's goldens, GLM and tool-parse tests, and also compilestensorfold-native's root.3f392a68brings DeepSeek-V4.1 L11d: batched rows, slots and round execution #699's fix:TF_DSV41_SESSION_RESIDENT/_RESIDENT_GIBdefault to unlimited (null). sessions: session memory for paged-KV families (page pool, prefix trie, RAM residency budget, streamed NVMe tier) (DeepSeek-V4.1 stack 4/14) #637 made0mean "no idle residency".Commits in this layer:
f5db10bf(Jay Leaton), and the mergesa0d81817and3f392a68.Receipt
3f392a681f3bae7422ea19407a6ecc74df079d5f, Linux x86_64, Zig 0.17.0. No GPU or checkpoint.3f392a68):test-server-cpuwith the new test indeepseek_test.zig:3f392a68: zig, fmt, desktop, mobile: SUCCESS.mainhas since moved to0a591781; that change touchesdocs/community-benchmarks.mdonly, which this branch does not touch.-Dnvcc.Checklist
"draft": false, a resumed prompt equals a fresh one, and concurrent equals solo.Merge Danger
Door: two-way. Reverting restores #699 without a data migration.
Blast Radius: server. Only checkpoints with
model_type: deepseek_v41take the family path, and other families keep the template path. The shared server files change for everyone (thejsonmodule wiring, prompt and generation hooks behindsrv.family), so a regression there would reach every family's HTTP serving.