RC4.1: faster prose and code decode, bit-exact with RC2; long GLM tool calls fixed

README.md

@@ -28,7 +28,8 @@ cp .env.example .env      # set WORKER to the other Spark's ssh destination
 
 Measured on two DGX Sparks joined by one 200G cable, with images built from
 a fresh clone. Receipts and caveats:
-[this release: KV shard, deterministic verify, faster MoE decode](results/2026-10-09-rc2/RESULT.md),
+[this release: faster single-stream decode, bit-exact with RC2](results/2026-10-12-rc4/RESULT.md),
+[RC2: KV shard, deterministic verify, faster MoE decode](results/2026-10-09-rc2/RESULT.md),
 [the display carveout's KV placement](results/2026-10-05-carveout-placement/RESULT.md),
 [a faster verify step and cheaper warm turns](results/2026-09-30-decode-step/RESULT.md),
 [RigMark decode, prefill and staggered arrivals on 2026-10-05](results/2026-10-05-rigmark/RESULT.md),
@@ -50,14 +51,14 @@ and [the first Atlas release](results/2026-09-29-atlas-merged/RESULT.md)
 | New session sharing a 24K system prompt with earlier ones: time to first token | **15.7 s** | 36.2 s with the cache policy off |
 | Idle session resuming after another session's 17 turns (median) | **2.5 s** | 49.6 s with the cache policy off |
 | Cold 61K / 125K prompt: time to first token | **25.0 / 49.1 s** | 29.2 / 62.5 s two engines ago |
-| Matrix: C1/C2 at 16K and 32K, C4 at 16K, 400 tokens each, cold (sum of walls) | **136.3 s** | vLLM SparkGLM 206.6 s · Mia TensorFold v1.10 166.9 s |
-| Staggered C4: four ~16K requests arriving 1 s apart, cold (median of 5) | **49.9 s** | vLLM SparkGLM 70.5 s · Mia TensorFold v1.10 60.2 s · Mia EXL3 113.2 s |
-| Staggered C4: four ~32K requests arriving 1 s apart, cold (median of 3) | **77.3 s** | Mia TensorFold v1.10 100.1 s |
-| Single-stream decode, 12 prose / 12 code prompts × 256 tokens | **42.9 / 61.7 tok/s** | 41.2 / 61.8 on 2026-10-09 morning; Mia TensorFold v1.10 is faster here ([release notes](results/2026-10-09-rc2/RESULT.md)) |
-| RigMark decode, code / prose / structured (thinking on, low effort) | **65.5 / 36.5 / 88.8 tok/s** | [RiNGSiDE](https://github.com/othexmr/GLM-5.3-Flash-NVFP4-2x-4x-DGX-Sparks-RiNGSiDE) vLLM TP2 (published) 56.5 / 33.0 / 83.4 |
-| RigMark cold prefill 8K / 32K / 64K: time to first token | **3.29 / 12.47 / 23.72 s** | RiNGSiDE 3.56 / 12.94 / 25.66 s |
+| Matrix: C1/C2 at 16K and 32K, C4 at 16K, 400 tokens each, cold (sum of walls) | **132.1 s** | vLLM SparkGLM 206.6 s · Mia TensorFold v1.10 166.9 s |
+| Staggered C4: four ~16K requests arriving 1 s apart, cold (median of 5) | **49.5 s** | vLLM SparkGLM 70.5 s · Mia TensorFold v1.10 60.2 s · Mia EXL3 113.2 s |
+| Staggered C4: four ~32K requests arriving 1 s apart, cold (median of 3) | **76.3 s** | Mia TensorFold v1.10 100.1 s |
+| Single-stream decode, 12 prose / 12 code prompts × 256 tokens | **45.6 / 64.9 tok/s** | 42.8 / 61.4 with RC2 (2026-10-09), same output bits ([release notes](results/2026-10-12-rc4/RESULT.md)) |
+| RigMark decode, code / prose / structured (thinking on, low effort) | **72.3 / 38.3 / 91.7 tok/s** | [RiNGSiDE](https://github.com/othexmr/GLM-5.3-Flash-NVFP4-2x-4x-DGX-Sparks-RiNGSiDE) vLLM TP2 (published) 56.5 / 33.0 / 83.4 |
+| RigMark cold prefill 8K / 32K / 64K: time to first token | **3.29 / 12.47 / 23.91 s** | RiNGSiDE 3.56 / 12.94 / 25.66 s |
 | RigMark staggered arrivals, prefill first: newcomer time to first token at 2 / 4 | **2.80 / 3.54 s** | RiNGSiDE 4.70 / 5.15 s |
-| RigMark short code, 1 / 4 streams, aggregate | **47.7 / 69.5 tok/s** | RiNGSiDE 44.0 at 1 stream; 97.3 at 6 streams (our 4 × 1M profile serves 4 at a time) |
+| RigMark short code, 1 / 4 streams, aggregate | **47.0 / 70.9 tok/s** | RiNGSiDE 44.0 at 1 stream; 97.3 at 6 streams (our 4 × 1M profile serves 4 at a time) |
 | Returning to a ~209K-token conversation evicted to disk (prefix cache on disk, 48 GB) | **1.2–1.5 s** | about 82 s without it |
 | Strict JSON (`response_format` json_schema), ~2,000-token structured answer | **30–50 s**, schema-valid | 135–155 s with masked serial decode |
 | Strict JSON object with 96 required keys, temperature 0 | **39 s**, 59 tok/s, valid | engine killed (out of memory) two releases ago |
@@ -254,17 +255,15 @@ comparison videos.
 
 ## The engine
 
-Atlas here is [`Enntity/atlas`](https://github.com/Enntity/atlas) commit
-`f2b805e7` on `sparkglm/atlas-20261009-rc2` (pinned in
-[`install/atlas-source.json`](install/atlas-source.json)). Its tree is
-identical to `sparkglm/atlas-20261009-rc2-layered` at `fea1ef6c` (tree
-`513bf0c7`), which is built from three
+Atlas here is [`Enntity/atlas`](https://github.com/Enntity/atlas) branch
+`sparkglm/atlas-20261012-rc4.1-layered` at `301479a2` (tree `ac5a080b`, pinned in
+[`install/atlas-source.json`](install/atlas-source.json)), built from three
 layers:
 
 1. Atlas-Inf `main` base at `6a3d24ec`.
 2. GLM-5.3-Flash support and optimizations (`upstream/glm53-flash`, at this
-   release `upstream/glm53-flash-20261009-rc2`), which we intend to propose
-   to Atlas-Inf after review. It started as
+   release `upstream/glm53-flash` at `0b515eec` plus the tool-value fix
+   `ade7b012`), which we intend to propose to Atlas-Inf after review. It started as
    Reiner Schmidt's port
    ([Mango-kid/atlas](https://github.com/Mango-kid/atlas/tree/feat/glm53-dual-spark));
    the engine's `docs/porting/GLM_5_3_FLASH.md` has the history.
@@ -273,11 +272,9 @@ layers:
    display carveout (including where the split KV pools are placed in it).
 
 These are published Enntity release snapshots, separate from current
-Atlas-Inf `main`. The manifest's top-level commit and tree identify the
-measured engine; its `layers` entries record the earlier integration history.
-The tree-equivalent layered reconstruction uses the RC2 contribution series
-at `a1c9b1bc`. The manifest stays unchanged so the install tree still names
-the measured image.
+Atlas-Inf `main`. The pinned tree is RC4's measured tree (`exp/prose-rc4` at
+`c6089633`) plus the GLM tool-value fix and four CI-only upstream commits; the
+manifest's `layers` entries record the layering.
 
 ## History
 

docs/INSTALL-MANUAL.md

@@ -5,7 +5,7 @@ time. Run steps 1–4 on **both** Sparks with the same `MODEL_ROOT`.
 
 | Component | Pin |
 |---|---|
-| Engine | [`Enntity/atlas`](https://github.com/Enntity/atlas) `sparkglm/atlas-20261009-rc2` @ `f2b805e7` ([`install/atlas-source.json`](../install/atlas-source.json)) |
+| Engine | [`Enntity/atlas`](https://github.com/Enntity/atlas) `sparkglm/atlas-20261012-rc4.1-layered` @ `301479a2` ([`install/atlas-source.json`](../install/atlas-source.json)) |
 | Model | [`nvidia/GLM-5.3-Flash-NVFP4`](https://huggingface.co/nvidia/GLM-5.3-Flash-NVFP4) @ `423acf37583782c51c142d145aef733d72943d93` (MIT) |
 | Drafter | [`incoai/GLM-5.3-Flash-DFlash2`](https://huggingface.co/incoai/GLM-5.3-Flash-DFlash2) @ `7d74cdd881ed7e32c31175984a67823127b66cfe` (CC BY-NC-ND 4.0) |
 | Native dependencies | FlashInfer `8eccd0c1`, CUTLASS `cf064d2e`, FlashKDA ([`install/flash_kda/`](../install/flash_kda/)), Rust 1.93.1, CUDA 13.0 |

install/atlas-source.json

@@ -2,24 +2,24 @@
   "schema": "sparkglm.atlas-engine-source/v2",
   "note": "The Atlas engine is built from the public Enntity/atlas fork at this exact commit; build.sh refuses any other tree.",
   "repository": "https://github.com/Enntity/atlas.git",
-  "branch": "sparkglm/atlas-20261009-rc2",
-  "commit": "f2b805e73d36a7e5527622e24f2f8199052e01e6",
-  "tree": "513bf0c7187cc202a54980533fb4c5325ec72b2e",
+  "branch": "sparkglm/atlas-20261012-rc4.1-layered",
+  "commit": "301479a20612c91243266a3fc7eea3100b99ad59",
+  "tree": "ac5a080b608b6cbdf420db63d76906ea450392dd",
   "layers": {
     "atlas_inf_main_base": "6a3d24ecb891adac1fac8edeef92de29bb6f3e15",
     "upstream_candidate": {
-      "branch": "upstream/glm53-flash-20261009",
-      "commit": "576ad34e",
-      "note": "upstream/glm53-flash-20261007-strings (GLM tool arguments keep their schema types; no effort header on the empty think block) plus copy (prompt-lookup) drafts beside DFlash2 (ATLAS_DFLASH_COPY_DRAFTS), the bare role-literal strip limited to ChatML models (GLM streamed replies lost the words tool/user/assistant), and Poolside tool-call headers held until the call validates."
+      "branch": "fix/glm-tool-value-markers",
+      "commit": "ade7b01281b422d2fefb16205685aabc3d46d9f4",
+      "note": "upstream/glm53-flash at 0b515eec (upstream/glm53-flash-20261012-rc4: the RC4 engine commits on a1c9b1bc plus four CI-only fixes, none changing GB10 runtime behavior) plus the GLM tool-value fix: the envelope-stuck guard (MAX_TOOL_BODY_TOKENS = 1024) recognized parameter values only by Qwen's <parameter=...> token ids, so it cut any GLM tool call longer than about 1024 tokens (finish_reason length, no tool call); GLM's single-token <arg_value>/</arg_value> are now resolved at startup and value bodies tracked by them."
     },
     "sparkglm_only": {
-      "commit": "72588541a466f53cb9d38afd74c515e14ab601db",
-      "note": "The unchanged FlashKDA/native sparse-MLA prefill bridges and GB10 display-carveout KV placement from fdd2f965, then the carveout takes latent KV pools only (ATLAS_KV_CARVEOUT_ORDER, default latent): the GPU maps the carveout L2-uncached, and sparse-index buffers there slowed 64K cold prefill by 7% (Enntity/sparkglm#30). Fork-only; used only with DISPLAY_CARVEOUT=1. Plus: sharded latent pools (ATLAS_GLM_KV_SHARD, opt-in through SPARKGLM_KV_SHARD) size and place in the display carveout."
+      "commit": "301479a20612c91243266a3fc7eea3100b99ad59",
+      "note": "The seven SparkGLM-only commits re-layered on the upstream candidate: FlashKDA/native sparse-MLA prefill bridges (the first, 661a26a0, now declares the KV-shard flag its native bridge reads, which upstream dropped as unused), GB10 display-carveout KV placement (latent pools only, ATLAS_KV_CARVEOUT_ORDER), and sharded latent pools (ATLAS_GLM_KV_SHARD, opt-in through SPARKGLM_KV_SHARD) sized and placed in the carveout. Fork-only; the carveout is used only with DISPLAY_CARVEOUT=1."
     },
     "integration": {
-      "branch": "sparkglm/atlas-20261009-rc2",
-      "commit": "b85befc6 + 6",
-      "note": "sparkglm/atlas-20261009-final plus: the canonical verify (exp/canonical-verify: one row-invariant kernel family per op for every verify width, plain decode as a one-row verify, dense FFN included, so greedy output no longer depends on the DFlash width, the drafter or concurrency; ATLAS_GLM_CANONICAL_VERIFY, default on), persistent prefetching MoE decode kernels (exp/moe-decode-bw, bit-identical; ATLAS_GLM_MOE_DECODE_PERSIST, on in the profiles) and one-launch KDA record commits (exp/kda-commit-batch, bit-identical), cherry-picked with -x."
+      "branch": "sparkglm/atlas-20261012-rc4-layered",
+      "commit": "5c523903c886d1abf2f4bda6a0af1cdf91c28523",
+      "note": "RC4 (tree ae5e79f8, the measured exp/prose-rc4 c6089633). This pin differs from it by the tool-value fix (spark-server only) and the four CI-only upstream commits; the decode path is unchanged."
     }
   },
   "kernel_target": "glm-5.3-flash",

install/profiles/4x1m.json

@@ -183,7 +183,18 @@
   "ATLAS_USE_HF_REF_JSON_DUMPS": "1",
   "ATLAS_DFLASH_COPY_DRAFTS": "1",
   "ATLAS_DFLASH_COPY_REPLY_MATCH": "16",
-  "ATLAS_DFLASH_COPY_MISS_MAX": "3"
+  "ATLAS_DFLASH_COPY_MISS_MAX": "3",
+  "ATLAS_DFLASH_CONF_PROBE": "256",
+  "ATLAS_RDMA_ONESHOT_SM_STAGE": "1",
+  "ATLAS_GLM_MOE_ROUTE_FUSE": "1",
+  "ATLAS_GLM_DENSE_TP_SPLIT": "1",
+  "ATLAS_GLM_KDA_COMMIT_CTAS": "96",
+  "ATLAS_GLM_DRAFT_TP_TOPK": "1",
+  "ATLAS_DFLASH_ASYNC_UPLOADS": "1",
+  "ATLAS_GLM_MLA_FUSE": "1",
+  "ATLAS_GLM_MXFP8_TILED": "1",
+  "ATLAS_DFLASH2_EDGE": "0.6",
+  "ATLAS_DFLASH_CONV_TWIN": "nvfp4"
  },
  "server_argv": [
   "serve",

install/profiles/4x512k.json

@@ -183,7 +183,18 @@
   "ATLAS_USE_HF_REF_JSON_DUMPS": "1",
   "ATLAS_DFLASH_COPY_DRAFTS": "1",
   "ATLAS_DFLASH_COPY_REPLY_MATCH": "16",
-  "ATLAS_DFLASH_COPY_MISS_MAX": "3"
+  "ATLAS_DFLASH_COPY_MISS_MAX": "3",
+  "ATLAS_DFLASH_CONF_PROBE": "256",
+  "ATLAS_RDMA_ONESHOT_SM_STAGE": "1",
+  "ATLAS_GLM_MOE_ROUTE_FUSE": "1",
+  "ATLAS_GLM_DENSE_TP_SPLIT": "1",
+  "ATLAS_GLM_KDA_COMMIT_CTAS": "96",
+  "ATLAS_GLM_DRAFT_TP_TOPK": "1",
+  "ATLAS_DFLASH_ASYNC_UPLOADS": "1",
+  "ATLAS_GLM_MLA_FUSE": "1",
+  "ATLAS_GLM_MXFP8_TILED": "1",
+  "ATLAS_DFLASH2_EDGE": "0.6",
+  "ATLAS_DFLASH_CONV_TWIN": "nvfp4"
  },
  "server_argv": [
   "serve",

install/profiles/8x128k.json

@@ -182,7 +182,18 @@
   "ATLAS_USE_HF_REF_JSON_DUMPS": "1",
   "ATLAS_DFLASH_COPY_DRAFTS": "1",
   "ATLAS_DFLASH_COPY_REPLY_MATCH": "16",
-  "ATLAS_DFLASH_COPY_MISS_MAX": "3"
+  "ATLAS_DFLASH_COPY_MISS_MAX": "3",
+  "ATLAS_DFLASH_CONF_PROBE": "256",
+  "ATLAS_RDMA_ONESHOT_SM_STAGE": "1",
+  "ATLAS_GLM_MOE_ROUTE_FUSE": "1",
+  "ATLAS_GLM_DENSE_TP_SPLIT": "1",
+  "ATLAS_GLM_KDA_COMMIT_CTAS": "96",
+  "ATLAS_GLM_DRAFT_TP_TOPK": "1",
+  "ATLAS_DFLASH_ASYNC_UPLOADS": "1",
+  "ATLAS_GLM_MLA_FUSE": "1",
+  "ATLAS_GLM_MXFP8_TILED": "1",
+  "ATLAS_DFLASH2_EDGE": "0.6",
+  "ATLAS_DFLASH_CONV_TWIN": "nvfp4"
  },
  "server_argv": [
   "serve",