perf(ds4): fuse expert-major MoE route combine

[cheese-cakee]

Summary

Speeds up DeepSeek-V4 heterogeneous prefill by fusing the post-MMID route reduction used by the production expert-major path.

The active path previously evaluated:

down_e -> mul(route weights) -> permute -> cont -> sum_rows -> add(shared expert)

This PR replaces the reduction and optional shared-expert add with one GGML_OP_DS4_MOE_COMBINE kernel. It now applies inside eval_moe_owner_expert_major_batched, so both the R9700 hot/shared owner and the Strix cold owner use it.

This is deliberately described as a post-MMID combine, not a down-projection epilogue: the grouped down projection still materializes down_e. The optimization removes the weighted-route intermediate and the following transpose/reduction chain.

Set DFLASH_MOE_FUSED_COMBINE=1 to enable the qualified expert-major path. Set it to 0 to retain the legacy graph sequence.

Heterogeneous prefill results

Measured on Lucebox6 with Radeon AI PRO R9700 gfx1201 + Strix Halo gfx1151, the ROCmFP2 model, unchanged production placement (hot/layer=60, hot stack 2580, cold stack 8428), caches disabled, and the same candidate binary with the feature forced OFF or ON.

Each cell used two warmups followed by five measured cache-cold requests. The order was OFF-A, ON-A, ON-B, OFF-B. All measured samples were retained.

Workload	OFF pooled median ± MAD	ON pooled median ± MAD	Improvement	FFN median change
401-token prefill, chunk=512	188.31 ± 0.45 tok/s	198.23 ± 2.15 tok/s	+5.27%	1626.75 → 1515.00 ms (−6.87%)
2048-token prefill, chunk=2048	308.39 ± 6.08 tok/s	339.93 ± 4.13 tok/s	+10.23%	4119.10 → 3597.70 ms (−12.66%)

Both bracketed comparisons agree in sign:

• 401: +6.09% in pair A and +5.00% in pair B; OFF bracket drift was +0.07%.
• 2048: +12.46% in pair A and +7.21% in pair B; OFF bracket drift was +3.90%.

The 2048 workload was noisier, but the gain survived both independent ON/OFF brackets. No outlier was deleted when calculating the pooled result.

Raw measured prefill samples

Workload / cell	Five measured samples, tok/s	Cell median ± MAD
401 OFF-A	65.02, 63.17, 188.29, 188.92, 188.31	188.29 ± 0.63
401 ON-A	51.87, 108.88, 200.38, 199.76, 200.38	199.76 ± 0.62
401 ON-B	97.21, 89.41, 197.85, 198.61, 199.54	197.85 ± 1.69
401 OFF-B	189.70, 188.93, 188.43, 188.30, 188.03	188.43 ± 0.40
2048 OFF-A	298.83, 303.71, 304.37, 303.99, 300.13	303.71 ± 0.66
2048 ON-A	341.56, 267.85, 342.75, 342.83, 305.69	341.56 ± 1.27
2048 ON-B	311.90, 334.57, 338.29, 351.93, 341.82	338.29 ± 3.72
2048 OFF-B	315.55, 312.40, 313.37, 322.52, 330.52	315.55 ± 3.15

The tested server-code base was b9d37b32. Current main at publication is f5475131; the intervening commits only change documentation and assets, so the server code under test is unchanged. The replacement PR commit is based directly on f5475131.

Kernel-level measurement

The focused benchmark uses the model's exact hidden width (n_embd=4096) and top-k 6. It compares the actual legacy graph with the fused operation after two warmups and seven measured samples.

Device	Tokens	Legacy median	Fused median	Combine speedup
R9700 gfx1201	401	0.761 ms	0.055 ms	13.76×
R9700 gfx1201	2048	4.522 ms	0.467 ms	9.68×
Strix gfx1151	401	2.031 ms	0.239 ms	8.50×
Strix gfx1151	2048	11.023 ms	1.163 ms	9.48×

These are combine-kernel speedups, not whole-model speedups. The end-to-end gains are the pooled figures above.

Correctness and execution proof

• Production telemetry recorded exactly 86 fused launches for one 401-token prefill: 43 hot/shared-owner launches and 43 cold/no-shared-owner launches.
• Every strict timing request matched the expected output SHA-256 cd5cb9fb5ac3c4f4007e8b41d117da21622439cd05c1728f3e82f90e4f869dad.
• The direct combine differential covers top-k 4/6, token counts 1/3/33/401, shared and no-shared outputs, actual legacy-vs-fused CPU/HIP byte parity, masked routes with NaN-poisoned omitted lanes, and signed-zero parity. It passed on both gfx1201 and gfx1151.
• test_deepseek4_mmid_grouped_cuda: 72/72 cases passed on each GPU, including masked fused-zero coverage.
• test_deepseek4_unit: passed on each GPU.
• git diff --check: clean.
• No foreign KFD process was observed in any retained strict cell; KFD was clean after qualification.
• A fresh independent review of the final replacement diff found no remaining correctness issue after the signed-zero and legacy-differential fixes.

The qualified evidence is preserved on Lucebox6 at:

/home/cheese/pr658-active-path-final-r2-20260826T1624Z/qualified-evidence

Manifest SHA-256: 44f1a353b6d84cd67f8339796296537b7ee7e6b901b57b8b9553d4cb89a5639f

Scope

This PR claims a heterogeneous prefill improvement. It does not claim a decode speedup. A deeper fusion that avoids materializing down_e would require a separate grouped-MMID down-projection redesign and is outside this change.

[cubic-dev-ai]

All reported issues were addressed across 8 files

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[cubic-dev-ai]

Review completed against the latest diff

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[cubic-dev-ai]

All reported issues were addressed across 3 files (changes from recent commits).

_{You’re at about 90% of the monthly reviewed-line limit. You may want to disable incremental reviews to conserve quota. Reviews will continue until that limit is exceeded. If you need help avoiding interruptions, please contact contact@cubic.dev.}

<sub>Tip: Review your code locally with the cubic CLI to iterate faster.

Re-trigger cubic</sub>