From 16d594058a83ae2d46e6c8389c02a6cd3599f6ea Mon Sep 17 00:00:00 2001 From: Jason McCartney Date: Mon, 12 Oct 2026 14:28:53 -0700 Subject: [PATCH 1/2] RC4.2: ATLAS_LOOP_NO_SUPPRESS=1 in all profiles (no tool-call hard-mask on repeated calls) After three identical tool calls the loop detector hard-masked for a turn, and GLM wrote its intended action as prose, repeating one paragraph for 756 tokens (seen twice in production on RC4.1). The switch keeps the detection, logging, metrics and soft bias and skips only the mask. Engine pin unchanged (301479a2). Co-Authored-By: Claude Opus 5.5 --- README.md | 4 ++++ install/profiles/4x1m.json | 3 ++- install/profiles/4x512k.json | 3 ++- install/profiles/8x128k.json | 3 ++- results/2026-10-12-rc4/RESULT.md | 11 +++++++++++ 5 files changed, 21 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 1b4a6fe..86ea9b3 100644 --- a/README.md +++ b/README.md @@ -129,6 +129,10 @@ videos are supported. - `8x128k`: up to eight requests with up to 128K each, for more concurrent short work. +All three set `ATLAS_LOOP_NO_SUPPRESS=1`: when an agent repeats the same tool call, +the engine logs it but no longer masks tool calls for the next turn, which had +left the model repeating its intended action as prose. + The context limit applies to each request; it does not reserve that much KV for every concurrent slot. At the measured 2.36M-token pool size, four full 1M-token requests cannot fit together. Requests wait when the pool lacks room. diff --git a/install/profiles/4x1m.json b/install/profiles/4x1m.json index 7fcf694..ec83d96 100644 --- a/install/profiles/4x1m.json +++ b/install/profiles/4x1m.json @@ -194,7 +194,8 @@ "ATLAS_GLM_MLA_FUSE": "1", "ATLAS_GLM_MXFP8_TILED": "1", "ATLAS_DFLASH2_EDGE": "0.6", - "ATLAS_DFLASH_CONV_TWIN": "nvfp4" + "ATLAS_DFLASH_CONV_TWIN": "nvfp4", + "ATLAS_LOOP_NO_SUPPRESS": "1" }, "server_argv": [ "serve", diff --git a/install/profiles/4x512k.json b/install/profiles/4x512k.json index a7ddb5d..0132c89 100644 --- a/install/profiles/4x512k.json +++ b/install/profiles/4x512k.json @@ -194,7 +194,8 @@ "ATLAS_GLM_MLA_FUSE": "1", "ATLAS_GLM_MXFP8_TILED": "1", "ATLAS_DFLASH2_EDGE": "0.6", - "ATLAS_DFLASH_CONV_TWIN": "nvfp4" + "ATLAS_DFLASH_CONV_TWIN": "nvfp4", + "ATLAS_LOOP_NO_SUPPRESS": "1" }, "server_argv": [ "serve", diff --git a/install/profiles/8x128k.json b/install/profiles/8x128k.json index 8bfd950..70585e1 100644 --- a/install/profiles/8x128k.json +++ b/install/profiles/8x128k.json @@ -193,7 +193,8 @@ "ATLAS_GLM_MLA_FUSE": "1", "ATLAS_GLM_MXFP8_TILED": "1", "ATLAS_DFLASH2_EDGE": "0.6", - "ATLAS_DFLASH_CONV_TWIN": "nvfp4" + "ATLAS_DFLASH_CONV_TWIN": "nvfp4", + "ATLAS_LOOP_NO_SUPPRESS": "1" }, "server_argv": [ "serve", diff --git a/results/2026-10-12-rc4/RESULT.md b/results/2026-10-12-rc4/RESULT.md index 74a7b54..d03bed2 100644 --- a/results/2026-10-12-rc4/RESULT.md +++ b/results/2026-10-12-rc4/RESULT.md @@ -4,6 +4,17 @@ The image is `846ee9e1e501`. It runs engine `5c523903` on `sparkglm/atlas-202610 `upstream/glm53-flash-20261012-rc4` at `ad047638` plus the unchanged SparkGLM-only layer. Its tree (`ae5e79f8`) is identical to `exp/prose-rc4` at `c6089633`, the build measured below. +## RC4.2: no tool-call hard-mask on repeated calls + +RC4.2 changes one profile switch; the engine (`301479a2`, tree `ac5a080b`) and every receipt below are unchanged. + +- **The incident.** In production, an agent on RC4.1 ran the same `grep` three times and got the same result each time. + The engine's loop detector then hard-masked `` for the next turn. Unable to call a tool, the model wrote + the action it meant to take as prose and repeated the same paragraph for 756 tokens, twice within a minute. The mask + has been in Atlas since its May release. +- **The switch.** All three profiles now set `ATLAS_LOOP_NO_SUPPRESS=1`. The detector still detects, logs and meters + repeated calls and keeps its soft bias; only the hard mask on `` is skipped. + ## RC4.1 hotfix: long GLM tool calls The pin is now engine `301479a2` on `sparkglm/atlas-20261012-rc4.1-layered` (tree `ac5a080b`). It is RC4 plus one fix, From 93d220b058921f83c253a3a5544c44f141490989 Mon Sep 17 00:00:00 2001 From: Jason McCartney Date: Mon, 12 Oct 2026 15:27:51 -0700 Subject: [PATCH 2/2] RC4.2: engine sparkglm/atlas-20261012-rc4.2-layered, a repeat notice instead of masking tool calls Drop ATLAS_LOOP_NO_SUPPRESS from the profiles (they equal RC4.1's again) and pin dfb9c26a (tree 89e392a3): on repeated identical tool rounds the engine appends a and thinks at xhigh on that turn, with open and unbiased. RESULT.md RC4.2 section and README note rewritten. Co-Authored-By: Claude Opus 5.5 --- README.md | 20 +++++++++++--------- docs/INSTALL-MANUAL.md | 2 +- install/atlas-source.json | 18 +++++++++--------- install/profiles/4x1m.json | 3 +-- install/profiles/4x512k.json | 3 +-- install/profiles/8x128k.json | 3 +-- results/2026-10-12-rc4/RESULT.md | 29 +++++++++++++++++++---------- 7 files changed, 43 insertions(+), 35 deletions(-) diff --git a/README.md b/README.md index 86ea9b3..2ac9097 100644 --- a/README.md +++ b/README.md @@ -121,6 +121,11 @@ Streaming, tool calls, JSON and structured output, reasoning controls (`chat_template_kwargs.enable_thinking`, `reasoning_effort`), images and short videos are supported. +When an agent repeats the same tool call and gets the same result, the engine +adds a repeat notice to that result and lets the model think harder on that +turn; it no longer blocks tool calls, which had left the model narrating the +action it meant to take ([RC4.2](results/2026-10-12-rc4/RESULT.md)). + **Profiles** (`PROFILE` in `.env`): - `4x1m` (default): up to four requests with up to 1,048,576 tokens each, the @@ -129,10 +134,6 @@ videos are supported. - `8x128k`: up to eight requests with up to 128K each, for more concurrent short work. -All three set `ATLAS_LOOP_NO_SUPPRESS=1`: when an agent repeats the same tool call, -the engine logs it but no longer masks tool calls for the next turn, which had -left the model repeating its intended action as prose. - The context limit applies to each request; it does not reserve that much KV for every concurrent slot. At the measured 2.36M-token pool size, four full 1M-token requests cannot fit together. Requests wait when the pool lacks room. @@ -260,14 +261,15 @@ comparison videos. ## The engine Atlas here is [`Enntity/atlas`](https://github.com/Enntity/atlas) branch -`sparkglm/atlas-20261012-rc4.1-layered` at `301479a2` (tree `ac5a080b`, pinned in +`sparkglm/atlas-20261012-rc4.2-layered` at `dfb9c26a` (tree `89e392a3`, pinned in [`install/atlas-source.json`](install/atlas-source.json)), built from three layers: 1. Atlas-Inf `main` base at `6a3d24ec`. 2. GLM-5.3-Flash support and optimizations (`upstream/glm53-flash`, at this - release `upstream/glm53-flash` at `0b515eec` plus the tool-value fix - `ade7b012`), which we intend to propose to Atlas-Inf after review. It started as + release `upstream/glm53-flash-20261012-rc4.2` at `d8a55e0f`: `0b515eec` + plus the tool-value fix and the repeat notice), which we intend to propose + to Atlas-Inf after review. It started as Reiner Schmidt's port ([Mango-kid/atlas](https://github.com/Mango-kid/atlas/tree/feat/glm53-dual-spark)); the engine's `docs/porting/GLM_5_3_FLASH.md` has the history. @@ -277,8 +279,8 @@ layers: These are published Enntity release snapshots, separate from current Atlas-Inf `main`. The pinned tree is RC4's measured tree (`exp/prose-rc4` at -`c6089633`) plus the GLM tool-value fix and four CI-only upstream commits; the -manifest's `layers` entries record the layering. +`c6089633`) plus the GLM tool-value fix, the repeat notice and four CI-only +upstream commits; the manifest's `layers` entries record the layering. ## History diff --git a/docs/INSTALL-MANUAL.md b/docs/INSTALL-MANUAL.md index f89a5d0..1ca209e 100644 --- a/docs/INSTALL-MANUAL.md +++ b/docs/INSTALL-MANUAL.md @@ -5,7 +5,7 @@ time. Run steps 1–4 on **both** Sparks with the same `MODEL_ROOT`. | Component | Pin | |---|---| -| Engine | [`Enntity/atlas`](https://github.com/Enntity/atlas) `sparkglm/atlas-20261012-rc4.1-layered` @ `301479a2` ([`install/atlas-source.json`](../install/atlas-source.json)) | +| Engine | [`Enntity/atlas`](https://github.com/Enntity/atlas) `sparkglm/atlas-20261012-rc4.2-layered` @ `dfb9c26a` ([`install/atlas-source.json`](../install/atlas-source.json)) | | Model | [`nvidia/GLM-5.3-Flash-NVFP4`](https://huggingface.co/nvidia/GLM-5.3-Flash-NVFP4) @ `423acf37583782c51c142d145aef733d72943d93` (MIT) | | Drafter | [`incoai/GLM-5.3-Flash-DFlash2`](https://huggingface.co/incoai/GLM-5.3-Flash-DFlash2) @ `7d74cdd881ed7e32c31175984a67823127b66cfe` (CC BY-NC-ND 4.0) | | Native dependencies | FlashInfer `8eccd0c1`, CUTLASS `cf064d2e`, FlashKDA ([`install/flash_kda/`](../install/flash_kda/)), Rust 1.93.1, CUDA 13.0 | diff --git a/install/atlas-source.json b/install/atlas-source.json index 510739d..04bd174 100644 --- a/install/atlas-source.json +++ b/install/atlas-source.json @@ -2,24 +2,24 @@ "schema": "sparkglm.atlas-engine-source/v2", "note": "The Atlas engine is built from the public Enntity/atlas fork at this exact commit; build.sh refuses any other tree.", "repository": "https://github.com/Enntity/atlas.git", - "branch": "sparkglm/atlas-20261012-rc4.1-layered", - "commit": "301479a20612c91243266a3fc7eea3100b99ad59", - "tree": "ac5a080b608b6cbdf420db63d76906ea450392dd", + "branch": "sparkglm/atlas-20261012-rc4.2-layered", + "commit": "dfb9c26a95c96faa9f94c44e23343b0f2b4f9db6", + "tree": "89e392a3d19535bf153d4cabe5d7ffee1e378382", "layers": { "atlas_inf_main_base": "6a3d24ecb891adac1fac8edeef92de29bb6f3e15", "upstream_candidate": { - "branch": "fix/glm-tool-value-markers", - "commit": "ade7b01281b422d2fefb16205685aabc3d46d9f4", - "note": "upstream/glm53-flash at 0b515eec (upstream/glm53-flash-20261012-rc4: the RC4 engine commits on a1c9b1bc plus four CI-only fixes, none changing GB10 runtime behavior) plus the GLM tool-value fix: the envelope-stuck guard (MAX_TOOL_BODY_TOKENS = 1024) recognized parameter values only by Qwen's token ids, so it cut any GLM tool call longer than about 1024 tokens (finish_reason length, no tool call); GLM's single-token / are now resolved at startup and value bodies tracked by them." + "branch": "upstream/glm53-flash-20261012-rc4.2", + "commit": "d8a55e0f3fe10374272249da66c026563ad2b1a9", + "note": "upstream/glm53-flash at 0b515eec (the RC4 engine commits plus four CI-only fixes) plus the GLM tool-value fix ade7b012 (tool calls longer than 1024 tokens were cut: the envelope-stuck guard knew only Qwen's parameter markers) plus d8a55e0f: when the same tool calls return the same non-error result twice or more, the latest tool result gets a and that turn the xhigh thinking rung (unless the client set a budget), and stays open and unbiased; the loop detector no longer hard-masks , which had left GLM narrating its intended action as prose." }, "sparkglm_only": { - "commit": "301479a20612c91243266a3fc7eea3100b99ad59", - "note": "The seven SparkGLM-only commits re-layered on the upstream candidate: FlashKDA/native sparse-MLA prefill bridges (the first, 661a26a0, now declares the KV-shard flag its native bridge reads, which upstream dropped as unused), GB10 display-carveout KV placement (latent pools only, ATLAS_KV_CARVEOUT_ORDER), and sharded latent pools (ATLAS_GLM_KV_SHARD, opt-in through SPARKGLM_KV_SHARD) sized and placed in the carveout. Fork-only; the carveout is used only with DISPLAY_CARVEOUT=1." + "commit": "dfb9c26a95c96faa9f94c44e23343b0f2b4f9db6", + "note": "The seven SparkGLM-only commits re-layered on the upstream candidate (26812d35..dfb9c26a): FlashKDA/native sparse-MLA prefill bridges (the first, 26812d35, declares the KV-shard flag its native bridge reads, which upstream dropped as unused), GB10 display-carveout KV placement (latent pools only, ATLAS_KV_CARVEOUT_ORDER), and sharded latent pools (ATLAS_GLM_KV_SHARD, opt-in through SPARKGLM_KV_SHARD) sized and placed in the carveout. Fork-only; the carveout is used only with DISPLAY_CARVEOUT=1." }, "integration": { "branch": "sparkglm/atlas-20261012-rc4-layered", "commit": "5c523903c886d1abf2f4bda6a0af1cdf91c28523", - "note": "RC4 (tree ae5e79f8, the measured exp/prose-rc4 c6089633). This pin differs from it by the tool-value fix (spark-server only) and the four CI-only upstream commits; the decode path is unchanged." + "note": "RC4 (tree ae5e79f8, the measured exp/prose-rc4 c6089633). This pin differs from it by two spark-server changes (the tool-value fix and the repeat notice) and four CI-only upstream commits; the decode path is unchanged. RC4.1 was sparkglm/atlas-20261012-rc4.1-layered 301479a2." } }, "kernel_target": "glm-5.3-flash", diff --git a/install/profiles/4x1m.json b/install/profiles/4x1m.json index ec83d96..7fcf694 100644 --- a/install/profiles/4x1m.json +++ b/install/profiles/4x1m.json @@ -194,8 +194,7 @@ "ATLAS_GLM_MLA_FUSE": "1", "ATLAS_GLM_MXFP8_TILED": "1", "ATLAS_DFLASH2_EDGE": "0.6", - "ATLAS_DFLASH_CONV_TWIN": "nvfp4", - "ATLAS_LOOP_NO_SUPPRESS": "1" + "ATLAS_DFLASH_CONV_TWIN": "nvfp4" }, "server_argv": [ "serve", diff --git a/install/profiles/4x512k.json b/install/profiles/4x512k.json index 0132c89..a7ddb5d 100644 --- a/install/profiles/4x512k.json +++ b/install/profiles/4x512k.json @@ -194,8 +194,7 @@ "ATLAS_GLM_MLA_FUSE": "1", "ATLAS_GLM_MXFP8_TILED": "1", "ATLAS_DFLASH2_EDGE": "0.6", - "ATLAS_DFLASH_CONV_TWIN": "nvfp4", - "ATLAS_LOOP_NO_SUPPRESS": "1" + "ATLAS_DFLASH_CONV_TWIN": "nvfp4" }, "server_argv": [ "serve", diff --git a/install/profiles/8x128k.json b/install/profiles/8x128k.json index 70585e1..8bfd950 100644 --- a/install/profiles/8x128k.json +++ b/install/profiles/8x128k.json @@ -193,8 +193,7 @@ "ATLAS_GLM_MLA_FUSE": "1", "ATLAS_GLM_MXFP8_TILED": "1", "ATLAS_DFLASH2_EDGE": "0.6", - "ATLAS_DFLASH_CONV_TWIN": "nvfp4", - "ATLAS_LOOP_NO_SUPPRESS": "1" + "ATLAS_DFLASH_CONV_TWIN": "nvfp4" }, "server_argv": [ "serve", diff --git a/results/2026-10-12-rc4/RESULT.md b/results/2026-10-12-rc4/RESULT.md index d03bed2..685c788 100644 --- a/results/2026-10-12-rc4/RESULT.md +++ b/results/2026-10-12-rc4/RESULT.md @@ -4,16 +4,25 @@ The image is `846ee9e1e501`. It runs engine `5c523903` on `sparkglm/atlas-202610 `upstream/glm53-flash-20261012-rc4` at `ad047638` plus the unchanged SparkGLM-only layer. Its tree (`ae5e79f8`) is identical to `exp/prose-rc4` at `c6089633`, the build measured below. -## RC4.2: no tool-call hard-mask on repeated calls - -RC4.2 changes one profile switch; the engine (`301479a2`, tree `ac5a080b`) and every receipt below are unchanged. - -- **The incident.** In production, an agent on RC4.1 ran the same `grep` three times and got the same result each time. - The engine's loop detector then hard-masked `` for the next turn. Unable to call a tool, the model wrote - the action it meant to take as prose and repeated the same paragraph for 756 tokens, twice within a minute. The mask - has been in Atlas since its May release. -- **The switch.** All three profiles now set `ATLAS_LOOP_NO_SUPPRESS=1`. The detector still detects, logs and meters - repeated calls and keeps its soft bias; only the hard mask on `` is skipped. +## RC4.2: a repeat notice instead of blocking tool calls + +The pin is now engine `dfb9c26a` on `sparkglm/atlas-20261012-rc4.2-layered` (tree `89e392a3`): RC4.1 plus one +spark-server commit (`d8a55e0f`). The profiles and the decode path are unchanged, and so are the receipts below. + +- **The incident.** In production, an agent on RC4.1 ran the same `grep` three times and got the same result each + time. The engine's loop detector then hard-masked `` for the next turn. Unable to call a tool, the model + wrote the action it meant to take as prose and repeated the same paragraph for 756 tokens, twice within a minute. + The mask has been in Atlas since its May release. +- **The fix.** From the second identical round (the same calls with the same arguments, returning the same non-error + result), the engine appends a `` to the latest tool result and gives that turn the `xhigh` thinking + rung, unless the client set an explicit budget. `` stays open and unbiased: no mask, no negative bias. + Turning the mask off alone was not enough; the model then repeated the identical call. +- **Before and after, same reproduction** (the agent's loop replayed on the pair): RC4.1 answered with a prose + loop of about 4,000 tokens; RC4.2 answers with a new tool call. On the same build the long tool call returns + intact, the prompt-logprob hash is `246cd1331b41`, and decode speed is unchanged. +- **Thinking harder on every turn did not help.** Raising the thinking budget for every turn from 128 to 256 tokens + scored 11/13 on RigMark Quality both ways, but took 4.5× as long (2,204 s against 485 s) with 22× the reasoning + tokens. So RC4.2 raises thinking only on the stuck turn. ## RC4.1 hotfix: long GLM tool calls