[AzeezIsh]

Problem

A code_puppy agent session on Nemotron 3.5 Super VL, served by Atlas, had every large tool call cut off. create_file writes landed as the 11 bytes </parameter, and replace_in_file calls arrived without their replacements array.

Atlas caps the tool-call envelope at MAX_TOOL_BODY_TOKENS (1024, log tag tool_envelope_stuck) and exempts the tokens inside a <parameter=KEY> value. scheduler/emit_step.rs::update_tool_param_state found those values by matching five hard-coded Qwen3.6 token IDs (< 27, = 28, > 29, </ 510, parameter 15704). Nemotron's tokenizer uses different IDs (< 1060, = 1061, > 1062, >\n 1561, </ 1885, parameter 31960). So on Nemotron no value was ever recognised, and all file content counted against the 1024-token cap. The model itself is fine.

Fix

• New scheduler/tool_markers.rs builds the marker token classes once from the served tokenizer's vocabulary:
  • an opening <
  • parameter
  • tokens starting with =
  • tokens starting with >
  • tokens ending with </
• The classes are installed in tokenizer_runtime.rs, at startup and on model swap. The startup log names them.
• If a tokenizer lacks any class, both the value tracking and the cap switch off. --tool-max-tokens still bounds a runaway envelope.
• Qwen behaviour changes too, for the better. Qwen's BPE merges = into common key names (=file, =new, =re), which the exact-ID check also missed. Those values are now recognised as values.

Verification on a GB10 (sparky)

Setup:

• Both arms are built the same way: control origin/main e29809d and fix dfd3a80.
• Both are served with scripts/serve-nemotron-gb10.sh --mode atlas --spec g8 (nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4 + LiLiCorr drafter).
• Request: create_file + replace_in_file tools, temperature 0, a single-file HTML site written with create_file. Each request ran twice non-streaming and twice streaming (tool_calls deltas accumulated).

Fix-arm startup log: Tool parameter marker classes: < x2 parameter x1 =* x166 >* x79 *</ x34

arm	mode	runs	finish_reason	completion tokens	arguments	content	tool_envelope_stuck
main e29809d	non-streaming	2	tool_calls (salvaged)	1027	39 chars, {"file_path":"index.html","content":""}	empty	1 per request
main e29809d	streaming	2	length	1027	25 chars, not valid JSON	—	1 per request
fix dfd3a80	non-streaming	2	tool_calls	3240 (stop)	9,305 chars, valid JSON	8,894 chars, ends </html>	0
fix dfd3a80	streaming	2	tool_calls	3240 (stop)	9,305 chars, valid JSON	8,894 chars, ends </html>	0

Main: 4/4 cut at the 1024-token envelope cap (tool_envelope_stuck = 4 in total, one per request). Fix: 4/4 complete, with identical output across runs.

Unit tests: cargo test -p spark-server --bins tool_markers passes 8/8. The tests replay a 1500-token Nemotron write: it completes with the Nemotron markers, is cut with the old Qwen IDs, and stays uncut with no markers. A never-closed envelope is still cut.

Golden check (fix arm, serve-nemotron-gb10.sh --mode atlas --spec g8 --check): P1 PASS, P9 PASS, parallel_multiple_78 PASS. All byte-exact.

Qwen: the live Qwen repro was not run in this pass. The Qwen behaviour change (merged =key forms now recognised as values) is covered by the unit tests only.

Note on the 8192 ceiling: a first repro prompt asked for a site larger than 8192 tokens. On the fix arm those requests ran the full 8192 tokens without a single tool_envelope_stuck, then stopped at --tool-max-tokens (8192) before the model closed the call, with finish_reason length and empty salvaged arguments. That is the separate envelope bound working as designed, not this bug. The table above uses a prompt that fits.

🤖 Generated with Claude Code

[AzeezIsh]

Qwen leg (Qwen3.8-27B NVFP4, same box): inconclusive, no regression signal

Both arms were served bare-metal with the @atlas/qwen3.8-27b-nvfp4 recipe's serve line (from sparkrun --dry-run): main e29809d and this branch dfd3a80. They ran the same large create_file repro, 2× non-streaming and 2× streaming.

• Fix arm startup: Tool parameter marker classes: < x2 parameter x1 =* x394 >* x199 *</ x90. The classes resolve on the Qwen tokenizer.
• Both arms behaved identically:
  • Every request ended at length: the first after 3,073 tokens, the rest after 1,377.
  • 0 tool_envelope_stuck on either arm.
  • The salvaged arguments were empty (non-streaming 39 chars, streaming 25 chars).
• Those requests were stopped by something other than the 1024-token envelope cap and other than --tool-max-tokens (8192): neither cap fired. So this leg did not exercise the code path this PR changes.
• What it does show: no new cuts and no behaviour difference between main and the fix on Qwen under this recipe. Whatever ends Qwen-27B generations at 1,377 tokens under this serve line is a separate issue, and I haven't root-caused it.