Skip to content

Drafter release: the MTP drafter sees the real context; long prompts get first tokens in order - #11

Merged
data-angel merged 2 commits into
mainfrom
release/drafter-20261016
Oct 17, 2026
Merged

data-angel merged 2 commits into
mainfrom
release/drafter-20261016

Conversation

@data-angel

Copy link
Copy Markdown
Contributor

Pins Enntity/atlas sparkqwen/atlas-20261016-drafter @ 1aa8d99f (tree a04c2e03). The release turns on five settings in all three profiles:

  • ATLAS_QWEN4EXP_MTP_ABSORB
  • ATLAS_QWEN4EXP_MTP_PREFILL=16384
  • ATLAS_QWEN4EXP_MTP_BF16_ROUTER
  • ATLAS_MTP_DEEP_WINDOW=16
  • ATLAS_PREFILL_QUEUE_FIRST_CHUNK

Image tag (install tree): f399caa2fb5e.

What changed in the engine (docs/ENGINE.md#drafter-series):

  • Drafter context. The MTP drafter's KV started empty for every request and kept rows built from its own guesses. It is now rewritten from the target's verified rows after every verify, and filled over the prompt at the end of prefill. With 7 drafts per step, the share of steps whose 7th draft was accepted went from 0.44 to 0.81.
  • Admission order. A prompt longer than one prefill chunk queues its first chunk behind the prefills already in flight, so staggered long prompts get their first tokens in order. Prompts that fit in one chunk still run at admission.
  • Drafter router. The drafter's router runs in BF16; the target's stays NVFP4.
  • Depth decisions. The depth controller decides every 16 verifies instead of every 48.
  • W8A8 hardening. Prefill numerics are unchanged.

Measured through the image ./start.sh built from a fresh clone, same pair, clocks uncapped (results/2026-10-16-drafter/RESULT.md):

This release FP4-Local release
RigMark code / prose / structured (tok/s) 112.7 / 56.9 / 127.5 96.5 / 57.1 / 104.4
sparkDash one stream: structured / prose / code / JSON 134.0 / 85.4 / 108.3 / 109.3 111.9 / 81.8 / 101.5 / 95.7
Four ~18K prompts arriving 1 s apart: first tokens 4.7 / 8.4 / 12.1 / 15.8 s 16.2 / 15.9 / 15.7 / 15.4 s
Four new chats on one ~20K system prompt: first turn 5.8 / 6.1 / 6.4 / 6.7 s 5.8 / 7.6 / 9.5 / 11.3 s

Gates:

  • Greedy output is identical to a speculation-off, cache-off reference: 8/8 at C1, C4 and C8, and 8/8 on prefix hits.
  • Prompt-logprob hash 798fc6cc2641, the same as the FP4-Local release.
  • Quality 40/40 and 12/12, tools 21/21, agentic 20/20, RigMark 9/9.
  • The README, banner and social preview carry these numbers.

🤖 Generated with Claude Code

data-angel and others added 2 commits October 16, 2026 22:26
Pins Enntity/atlas sparkqwen/atlas-20261016-drafter (1aa8d99f, tree a04c2e03)
and turns on queued first chunks, the MTP drafter's KV over verified rows and
the prompt, the BF16 drafter router and the dynamic-depth decision window.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…MITATIONS and banners

The MTP drafter now sees the target's own rows for the prompt and every
verified token: RigMark code 96.5 -> 112.7 and structured 104.4 -> 127.5
tok/s, prose level, sparkDash one-stream up 4-14%. Long prompts arriving
together get their first tokens in order (4.7/8.4/12.1/15.8 s instead of
all about 16 s). Exact against a speculation-off, cache-off reference; same
prompt logprobs as the FP4-Local release.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@data-angel
data-angel merged commit c822cb6 into main Oct 17, 2026
1 check passed
@data-angel
data-angel deleted the release/drafter-20261016 branch October 17, 2026 05:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant