Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 11 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,11 @@ Streaming, tool calls, JSON and structured output, reasoning controls
(`chat_template_kwargs.enable_thinking`, `reasoning_effort`), images and short
videos are supported.

When an agent repeats the same tool call and gets the same result, the engine
adds a repeat notice to that result and lets the model think harder on that
turn; it no longer blocks tool calls, which had left the model narrating the
action it meant to take ([RC4.2](results/2026-10-12-rc4/RESULT.md)).

**Profiles** (`PROFILE` in `.env`):

- `4x1m` (default): up to four requests with up to 1,048,576 tokens each, the
Expand Down Expand Up @@ -256,14 +261,15 @@ comparison videos.
## The engine

Atlas here is [`Enntity/atlas`](https://github.com/Enntity/atlas) branch
`sparkglm/atlas-20261012-rc4.1-layered` at `301479a2` (tree `ac5a080b`, pinned in
`sparkglm/atlas-20261012-rc4.2-layered` at `dfb9c26a` (tree `89e392a3`, pinned in
[`install/atlas-source.json`](install/atlas-source.json)), built from three
layers:

1. Atlas-Inf `main` base at `6a3d24ec`.
2. GLM-5.3-Flash support and optimizations (`upstream/glm53-flash`, at this
release `upstream/glm53-flash` at `0b515eec` plus the tool-value fix
`ade7b012`), which we intend to propose to Atlas-Inf after review. It started as
release `upstream/glm53-flash-20261012-rc4.2` at `d8a55e0f`: `0b515eec`
plus the tool-value fix and the repeat notice), which we intend to propose
to Atlas-Inf after review. It started as
Reiner Schmidt's port
([Mango-kid/atlas](https://github.com/Mango-kid/atlas/tree/feat/glm53-dual-spark));
the engine's `docs/porting/GLM_5_3_FLASH.md` has the history.
Expand All @@ -273,8 +279,8 @@ layers:

These are published Enntity release snapshots, separate from current
Atlas-Inf `main`. The pinned tree is RC4's measured tree (`exp/prose-rc4` at
`c6089633`) plus the GLM tool-value fix and four CI-only upstream commits; the
manifest's `layers` entries record the layering.
`c6089633`) plus the GLM tool-value fix, the repeat notice and four CI-only
upstream commits; the manifest's `layers` entries record the layering.

## History

Expand Down
2 changes: 1 addition & 1 deletion docs/INSTALL-MANUAL.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ time. Run steps 1–4 on **both** Sparks with the same `MODEL_ROOT`.

| Component | Pin |
|---|---|
| Engine | [`Enntity/atlas`](https://github.com/Enntity/atlas) `sparkglm/atlas-20261012-rc4.1-layered` @ `301479a2` ([`install/atlas-source.json`](../install/atlas-source.json)) |
| Engine | [`Enntity/atlas`](https://github.com/Enntity/atlas) `sparkglm/atlas-20261012-rc4.2-layered` @ `dfb9c26a` ([`install/atlas-source.json`](../install/atlas-source.json)) |
| Model | [`nvidia/GLM-5.3-Flash-NVFP4`](https://huggingface.co/nvidia/GLM-5.3-Flash-NVFP4) @ `423acf37583782c51c142d145aef733d72943d93` (MIT) |
| Drafter | [`incoai/GLM-5.3-Flash-DFlash2`](https://huggingface.co/incoai/GLM-5.3-Flash-DFlash2) @ `7d74cdd881ed7e32c31175984a67823127b66cfe` (CC BY-NC-ND 4.0) |
| Native dependencies | FlashInfer `8eccd0c1`, CUTLASS `cf064d2e`, FlashKDA ([`install/flash_kda/`](../install/flash_kda/)), Rust 1.93.1, CUDA 13.0 |
Expand Down
18 changes: 9 additions & 9 deletions install/atlas-source.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,24 +2,24 @@
"schema": "sparkglm.atlas-engine-source/v2",
"note": "The Atlas engine is built from the public Enntity/atlas fork at this exact commit; build.sh refuses any other tree.",
"repository": "https://github.com/Enntity/atlas.git",
"branch": "sparkglm/atlas-20261012-rc4.1-layered",
"commit": "301479a20612c91243266a3fc7eea3100b99ad59",
"tree": "ac5a080b608b6cbdf420db63d76906ea450392dd",
"branch": "sparkglm/atlas-20261012-rc4.2-layered",
"commit": "dfb9c26a95c96faa9f94c44e23343b0f2b4f9db6",
"tree": "89e392a3d19535bf153d4cabe5d7ffee1e378382",
"layers": {
"atlas_inf_main_base": "6a3d24ecb891adac1fac8edeef92de29bb6f3e15",
"upstream_candidate": {
"branch": "fix/glm-tool-value-markers",
"commit": "ade7b01281b422d2fefb16205685aabc3d46d9f4",
"note": "upstream/glm53-flash at 0b515eec (upstream/glm53-flash-20261012-rc4: the RC4 engine commits on a1c9b1bc plus four CI-only fixes, none changing GB10 runtime behavior) plus the GLM tool-value fix: the envelope-stuck guard (MAX_TOOL_BODY_TOKENS = 1024) recognized parameter values only by Qwen's <parameter=...> token ids, so it cut any GLM tool call longer than about 1024 tokens (finish_reason length, no tool call); GLM's single-token <arg_value>/</arg_value> are now resolved at startup and value bodies tracked by them."
"branch": "upstream/glm53-flash-20261012-rc4.2",
"commit": "d8a55e0f3fe10374272249da66c026563ad2b1a9",
"note": "upstream/glm53-flash at 0b515eec (the RC4 engine commits plus four CI-only fixes) plus the GLM tool-value fix ade7b012 (tool calls longer than 1024 tokens were cut: the envelope-stuck guard knew only Qwen's parameter markers) plus d8a55e0f: when the same tool calls return the same non-error result twice or more, the latest tool result gets a <REPEAT NOTICE> and that turn the xhigh thinking rung (unless the client set a budget), and <tool_call> stays open and unbiased; the loop detector no longer hard-masks <tool_call>, which had left GLM narrating its intended action as prose."
},
"sparkglm_only": {
"commit": "301479a20612c91243266a3fc7eea3100b99ad59",
"note": "The seven SparkGLM-only commits re-layered on the upstream candidate: FlashKDA/native sparse-MLA prefill bridges (the first, 661a26a0, now declares the KV-shard flag its native bridge reads, which upstream dropped as unused), GB10 display-carveout KV placement (latent pools only, ATLAS_KV_CARVEOUT_ORDER), and sharded latent pools (ATLAS_GLM_KV_SHARD, opt-in through SPARKGLM_KV_SHARD) sized and placed in the carveout. Fork-only; the carveout is used only with DISPLAY_CARVEOUT=1."
"commit": "dfb9c26a95c96faa9f94c44e23343b0f2b4f9db6",
"note": "The seven SparkGLM-only commits re-layered on the upstream candidate (26812d35..dfb9c26a): FlashKDA/native sparse-MLA prefill bridges (the first, 26812d35, declares the KV-shard flag its native bridge reads, which upstream dropped as unused), GB10 display-carveout KV placement (latent pools only, ATLAS_KV_CARVEOUT_ORDER), and sharded latent pools (ATLAS_GLM_KV_SHARD, opt-in through SPARKGLM_KV_SHARD) sized and placed in the carveout. Fork-only; the carveout is used only with DISPLAY_CARVEOUT=1."
},
"integration": {
"branch": "sparkglm/atlas-20261012-rc4-layered",
"commit": "5c523903c886d1abf2f4bda6a0af1cdf91c28523",
"note": "RC4 (tree ae5e79f8, the measured exp/prose-rc4 c6089633). This pin differs from it by the tool-value fix (spark-server only) and the four CI-only upstream commits; the decode path is unchanged."
"note": "RC4 (tree ae5e79f8, the measured exp/prose-rc4 c6089633). This pin differs from it by two spark-server changes (the tool-value fix and the repeat notice) and four CI-only upstream commits; the decode path is unchanged. RC4.1 was sparkglm/atlas-20261012-rc4.1-layered 301479a2."
}
},
"kernel_target": "glm-5.3-flash",
Expand Down
20 changes: 20 additions & 0 deletions results/2026-10-12-rc4/RESULT.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,26 @@ The image is `846ee9e1e501`. It runs engine `5c523903` on `sparkglm/atlas-202610
`upstream/glm53-flash-20261012-rc4` at `ad047638` plus the unchanged SparkGLM-only layer. Its tree (`ae5e79f8`) is
identical to `exp/prose-rc4` at `c6089633`, the build measured below.

## RC4.2: a repeat notice instead of blocking tool calls

The pin is now engine `dfb9c26a` on `sparkglm/atlas-20261012-rc4.2-layered` (tree `89e392a3`): RC4.1 plus one
spark-server commit (`d8a55e0f`). The profiles and the decode path are unchanged, and so are the receipts below.

- **The incident.** In production, an agent on RC4.1 ran the same `grep` three times and got the same result each
time. The engine's loop detector then hard-masked `<tool_call>` for the next turn. Unable to call a tool, the model
wrote the action it meant to take as prose and repeated the same paragraph for 756 tokens, twice within a minute.
The mask has been in Atlas since its May release.
- **The fix.** From the second identical round (the same calls with the same arguments, returning the same non-error
result), the engine appends a `<REPEAT NOTICE>` to the latest tool result and gives that turn the `xhigh` thinking
rung, unless the client set an explicit budget. `<tool_call>` stays open and unbiased: no mask, no negative bias.
Turning the mask off alone was not enough; the model then repeated the identical call.
- **Before and after, same reproduction** (the agent's loop replayed on the pair): RC4.1 answered with a prose
loop of about 4,000 tokens; RC4.2 answers with a new tool call. On the same build the long tool call returns
intact, the prompt-logprob hash is `246cd1331b41`, and decode speed is unchanged.
- **Thinking harder on every turn did not help.** Raising the thinking budget for every turn from 128 to 256 tokens
scored 11/13 on RigMark Quality both ways, but took 4.5× as long (2,204 s against 485 s) with 22× the reasoning
tokens. So RC4.2 raises thinking only on the stuck turn.

## RC4.1 hotfix: long GLM tool calls

The pin is now engine `301479a2` on `sparkglm/atlas-20261012-rc4.1-layered` (tree `ac5a080b`). It is RC4 plus one fix,
Expand Down