[linson007]

Tencent's Qwen3.8-27B DSpine checkpoint is rejected by the DFlash2-only loader. This adds an opt-in native Metal backend through --draft-model: causal draft attention, learned bridge writes, parallel final-write refinement, and the conditional proposal probabilities consumed by the existing verifier. Cache restore and window sizing preserve DSpine's 8,192-token draft window. The native benchmark now honors that configured window as serving already does.

Experimental: the serving implementation uses eight rows of a checkpoint trained for sixteen. DFlash2 remains the default. Sixteen-row execution is included as reproducible research patches, with explicit rollout blockers.

Measured results

Apple M5 Pro, 20-core GPU, 48 GB, macOS 26.6; the same affine 4-bit Qwen3.8-27B target for both drafters. All 66 loaded target images are byte-identical. Draft backbones use affine 4-bit groups of 64; DSpine's bridge stays BF16. INT8 KV, FP32 recurrent state, ANE disabled, 32K configured context. DFlash2 retains its 2,048-token draft window; DSpine retains 8,192.

Comparison	Measured result	Scope/verdict
Parallel versus initial serial DSpine refinement	77.0 → 82.4 tok/s, +7.1%	Four coding prompts, HTTP solo ABBA; outputs and cycle counts unchanged, 1.1% spread
Serving DS8 versus DF8 on the parallel build	86.2 → 81.2 tok/s, −5.8% solo; −12.4% four-concurrent end-to-end	Earlier four-prompt comparison; exploratory workload, not a general ranking
Research DS16 versus DS8 with M16 policy	+16.19%, +15.32%, +16.04% native serial throughput	Three fixed ABBA groups, 30 primary short-prompt tasks plus a separate count control; all registered output/noise/thermal gates passed
Research DS16 versus DF8+M16, short native corpus	+18.3–18.9% aggregate TPS, descriptive	30 primary short-prompt tasks; all outputs match, but groups 1/2 fail per-case noise gates; not qualified
Research DS16 versus DF8+M16, long native corpus	+6.0–7.0% aggregate decode TPS	Two frozen repository-source contexts (8K prose / 16K code); all three registered groups pass; prose −3.5%, code +17–19%
Research DS16 versus DF8+M16, HTTP	Inconclusive in all three groups	11–46% endpoint spread and batch-width-dependent target outputs; all trials retained

The eight-hour study ran November 10, 03:18:48–11:18:48 UTC, with serial owned GPU jobs and continuous hardware observations. Power source varied: the DS16/DS8 first group had AC/battery transitions; the following two groups stayed on AC at recorded boundaries. Registered native gates did not require AC, so results retain that additional limitation. The final direct drafter comparisons started on battery; detailed round-boundary power observations accompany their verdicts. No noisy/failed group was dropped or repeated to obtain a pass.

DS16 reduces the primary cycle count from 1,303 to 1,062 at identical outputs. Its recorded wall per cycle rises about 5.9%, while tokens per cycle rise about 22.7%. Some prose requests regress by about 3–6%; fresh twelve-case output agreement and acceptance screens are separate from throughput qualification. Higher draft precision changes acceptance little here (4-bit 1,303 cycles, native BF16 planes 1,289), and does not reproduce the upstream BF16 arithmetic/training target.

On 67 matched short primary prefixes, mean first-cycle output counts are DF8 5.955, DS8 5.940, DS16 9.507; all 71 full continuations including controls match. DS8 wins eight, loses eight and ties 51 prefixes. These overlapping calibration prefixes do not establish a general acceptance gap or forecast TPS. Community feedback reports stronger DSpine acceptance on a different real-agent/reference setup; those external results are unverified here and motivate independent agent-history and conditional-probability parity checks.

At concurrency, 16 rows × four lanes exposes an expensive M64 gate/up path. The existing two-pass alternative is the simplest candidate to calibrate. Its descriptive HTTP gain did not qualify: two groups failed strict output equality. A same-binary diagnostic reproduces target output variation with physical batch width under both policies, while policy outputs agree at each fixed width. A separate forced target-logit probe shows an argmax flip at a small margin; it does not identify the exact original state or responsible operator. Smaller GDN storage and shared projection/gate-up experiments passed numerical checks but did not establish application throughput gains.

Validation and limits

• Shipping eight-row implementation: production/native builds, CPU and architecture checks, independent GPU-validated bridge/refinement equations at widths 1–4, and focused Python suites (33 tests). Initial installer/launcher checks: 285 tests (8 skipped) and 54 tests.
• Shipping serving: greedy/sampled/schema output, four concurrent requests, long-prefix reuse, disk restore, cancellation and follow-up generation; relevant checks also ran under Metal validation.
• Research sixteen-row candidate: 9,000 cold B1 sampled requests, 78 tests with no rejection after Holm correction over the reserved 234-test family. This is a finite early-token regression screen, not equivalence; no EOS termination was exercised.
• Research lifecycle: Metal-validated 15,928-token cold/warm/suffix/disk restore and actual priority suspension/replay in five cases, all complete output arrays matching. The separate synthetic memory-pressure probe did not trigger suspension.
• Fresh portable wide-policy rebuild, DSpine/draft-attention/plain16 GDN shader oracles and model smoke passed. The original prototype's full Table16 sixteen-row GDN test fails an out-of-bounds store. A later packing-only follow-up passes the full sixteen-row GDN suite under Metal validation, with unchanged tolerances. Register selection remains disabled; consumers still require explicit eight-row tile dispatch/fallback and separate qualification.

The long native comparison measures decode only: summed cold request-wall means improve just 0.91% descriptively (61.185→60.633 s), because 15–34 s prefill dominates. Its four original source prompts were narrowed to the 8K/16K pair before any of those prompts ran, to fit all three groups inside eight hours; both protocols are retained.

Next implementation: carry per-model geometry through planning, buffers, sampling and pipeline identities; adopt the packing repair and correct or reject unsupported register dispatch; account for about 260.7 MiB extra capacity only on enabled DSpine models; calibrate wide gate/up; then qualify batch-dependent width selection on new workloads. Global macro replacement and research environment variables are not proposed serving defaults.

Full eight-hour report and reproduction instructions · Result index · Evidence manifest · Earlier refinement results

A single unrotated vocabulary projection is required.

./splash serve --model mlx-community/Qwen3.8-27B-4bit \
  --draft-model tencent/Qwen3.8-27B-DSpine-b16

Refs #724.

[sady4850]

Hi, I was looking at DSpine for my M1 Pro setup some days ago, so I have some offline numbers which maybe help with the "acceptance gap" question you left open.

Setup. Teacher-forced paired test, both drafts on exactly the same anchors. Data is 43 real agent traces of Qwen3.8-27B, up to 32K context (40 Pi coding agent tasks + 3 of my own omp sessions). It is 117,712 blocks total, sampled in buckets 0-4K / 4-8K / 8-16K / 16-24K / 24-32K. Target Qwen3.8-27B on A100, run two times: bf16, and fake-quantized to MLX affine 4-bit g64 (like mlx-community 4bit). DFlash2 = incoai/Qwen3.8-27B-DFlash2 through its own NeMo forward. DSpine = tencent/Qwen3.8-27B-DSpine-b16 with greedy serial refinement, torch port, checked vs the reference dspine_mlx forward (59/60 tokens same). Accept = 1 + leading matches, 95% CI is bootstrap over sequences.

DSpine / DFlash2 accept (MLX4 target, bf16 target same within 0.01):

DSpine config	DSpine accept	ratio vs DFlash2 (4.03)	95% CI
8 rows, window 2048	4.32	x1.073	[1.066, 1.080]
8 rows, window 8192	4.49	x1.115	[1.107, 1.125]
16 rows, window 2048	5.50	x1.366	[1.335, 1.400]
16 rows, window 8192	5.90	x1.466	[1.424, 1.512]

With 8 rows / window 8192 (same as in this PR), DSpine is ahead in every bucket, between +10% and +14%. It is a bit smaller at long context, but it is never below DFlash2. Scoring against target argmax instead of the recorded tokens gives the same picture (x1.111). By position, DSpine conditional accept grows from 0.80 to 0.87 at positions 3-7. DFlash2 stays flat around 0.77.

4-bit draft. You wrote that BF16 draft precision is not measured. I checked it on M1 with MLX: DSpine draft backbone quantized affine 4-bit g64, bridge and codebook kept bf16 like in this PR, real MLX4 target, 7 sequences, 3840 blocks, same anchors for both:

config	bf16 draft	q4 draft	q4/bf16 [95% CI]
8 rows w2048	4.89	4.87	x0.998 [0.994, 1.003]
8 rows w8192	5.11	5.10	x0.998 [0.994, 1.003]
16 rows w8192	7.11	7.10	x0.997 [0.992, 1.004]

So quantizing the backbone is basically free for acceptance. I don't think it is the cause.

What this means for the PR numbers. Offline, 8-row DSpine should give about +11% accept over DFlash2, and here it is -5.8% tok/s. On M1 Pro I measured the cost side with decode-profile (B1 cycle ~177 ms at 4K). The DFlash2 draft is ~10.5% of the cycle, and draft attention grows linear with window: 2.2 ms at 2048 -> ~7.8 ms at 8192 extrapolated. The bridge is ~78 MB bf16 per step. Altogether I expect ~+7% net decode on M1 if acceptance holds, not a loss. On M5 Pro the ratio of draft to verify can be different, of course.