[job] Qwen3.8-Flash-Next on Strix Halo: prefill and TTFT

Status (updated 2026-10-01 19:30Z)

State: step 3 (prefix caching) is moved first, because the agentic data says it's the bigger lever. Steps 1–2 (cold prefill rate) still stand.

Measured on the Windows 1007-turn replay, Strix Halo Windows: Flash-Next port on main + MoE prefill transpose (2k TTFT ~2x) (supersedes #54) #80 tree, no prefix cache: TTFT median 64.5 s / p90 112 s vs 3.9 s of decode, so 92% of agentic wall time is re-prefill.
Draft strix-windows: Flash-Next launchers + prefix caching on by default (stacked on #80) #149: Windows Flash-Next recipe with prefix caching on, 4 Marconi slots (~0.45 GB). It fits by shrinking KV from ~97k to ~77k tokens. GB10 reference: [job] Qwen3.8-Flash-Next: fast multi-turn TTFT with prefix caching #69, 0.74 s warm vs 18.1 s cold at 27k.
Next: strix-windows: Flash-Next launchers + prefix caching on by default (stacked on #80) #149's winbox gates (poisoning gate → ST-995 → replay) right after Strix Halo Windows: Flash-Next port on main + MoE prefill transpose (2k TTFT ~2x) (supersedes #54) #80 merges. Then step 1's prefill profile at 2k/8k/32k, for first-turn and long-document TTFT.

Why: long prompts are the agentic workload, and time-to-first-token on Strix Halo is dominated by prefill. Measured on winbox (Windows 11, ROCm 10, VGM 32 GB), C=1:

	2k first token	≈ prefill rate	fingerprint
before #80's MoE transpose	—	70–75 tok/s at 2k and 7k	`fnext-win-r2`, #57 status
#80	9.56–9.66 s	≈ 210 tok/s	#80 `bf06515a7` / `c6efb5f60`, boot-7g recipe, 2026-09-28

The Windows recipe also runs with prefix caching off (--ssm-cache-slots 0), so every agentic turn re-prefills its full history.

Done bar (serve-amd.ps1 -Model qwen3.8-flash-next and the Linux equivalent):

prefill ≥ 1,000 tok/s at 8k and at 32k, i.e. TTFT ≤ 8 s at 8k and ≤ 33 s at 32k;
ppl, needle and kl_drift unchanged, plus ST-995 and the 1007-turn replay (AGENTS.md platform rule);
warm multi-turn TTFT: prefix caching with SSM snapshots on by default, inside the Windows commit wall (~84.5 GiB at VGM 32 GB).

Plan, each step gated by the one before:

Measure (no code): prefill profile at 2k / 8k / 32k. Split out the MoE grouped GEMM, the transpose, dequant, attention/QSA, the GDN chunk kernels and mHC. Report achieved TFLOP/s against the WMMA peak.
Move the MoE grouped GEMM and the dense projections onto WMMA tiles tuned for gfx1151 wave32. Keep a persisted per-shape plan cache for any BF16 shapes that go through hipBLASLt, so there's no online heuristic search.
Prefix caching: size the Marconi SSM snapshot pool to fit beside the KV budget and turn it on in the recipe. Measure warm TTFT on the agentic replay.

Related: #74 (GB10 long-context prefill), #72 (Strix 27B prefill, the w4a16_gemm_t_m128 finding), #69 (GB10 warm TTFT).

A measurement for step 3, from the 1007-turn Windows replay on #80 (winbox, 577F510D, no prefix cache): across the last 400 turns, TTFT median is 64.5 s (p90 112 s) and decode is 3.9 s. On the agentic workload, 92% of wall time is re-prefill.

So step 3 (prefix caching) is the larger lever on that workload than steps 1–2. The tail-snapshot restore turns a turn into "new tokens only": on GB10 #69 measured a 27k warm turn at 0.74 s vs 18.1 s cold. Cold prefill rate still matters for first turns and the ST/needle legs.

Memory: the live serve has 6061 KV blocks (~97k tokens), so 4 Marconi slots (~0.45 GB) fit. 16 slots wouldn't (KV would drop below 32k).
Draft: strix-windows: Flash-Next launchers + prefix caching on by default (stacked on #80) #149 makes that the Windows recipe default. Its gates are listed there.

Same root cause and fix as the 27B prefill comment on #72 (#72). For Flash-Next specifically, the routed-expert kernels are the worst offenders:

moe_w4a16_fused_gate_up_t and moe_w4a16_grouped_gemm_ptrtable_t run ~687 instructions per 32-K step for 16 WMMA (43/WMMA).
Each expert sees only ~40 rows (2,048 × 10 / 512), so the per-weight dequant is amortized over far fewer rows than the dense kernel's 128.
The F16 twins (exact dequant + the m128 pipeline) cut that to 267 instructions (16.7/WMMA).
Benchmark: routed gate+up (2×640×2560) and down (2560×640) at 2,048 tokens / top-10 / 512 experts, against an FP64 reference: strix job 144, plus the winbox slot after Strix Halo Windows: Flash-Next port on main + MoE prefill transpose (2k TTFT ~2x) (supersedes #54) #80's ST-995.

The F16-operand GEMM twins discussed above are now up as draft #155: dense m128, routed MoE and w8a16 FP8, plus engine wiring. 914/914 tests pass. It isn't merge-ready until the GPU bench (strix job 144, also built on winbox) shows the speedup and accuracy, and then the AMD gates.

Step 1 done: Windows Flash-Next prefill attribution.

Source: Claude's traced prefill on winbox, 2026-10-02 23:07Z.

Build: tree 93a99cc094 (the Strix Halo Windows: Flash-Next port on main + MoE prefill transpose (2k TTFT ~2x) (supersedes #54) #80 landing tree), spark 577F510D, Flash-Next NVFP4.
Settings: SERIAL=1, thinking off, ATLAS_TRACE_LAUNCH=1 + _SYNC=1.
Prompt: 9,302 tokens, so 5 chunks of ≤ 2,049.
Trace: 7,792 launches; 34.5 s summed GPU over a 51.1 s span.
Untraced TTFT for this prompt class: 37–43 s at 9,967 tokens (plan p4, n=3).

category	GPU s	share	note
routed MoE GEMMs (`moe_w4a16_*_t_k64`)	12.32	35.7%	33.0 ms gate_up + 18.4 ms down per layer-chunk
attention q/k/v/o (`fp8_gemm_t_m128`, `fp8_fp8_gemm_t_m128`)	7.39	21.4%	software E4M3 on gfx1151, ~1.7–2.4 TF/s
router (`w4a16_gemm`, N=512, K=2560)	3.42	9.9%	17.4 ms at M=2048 but 1.86 ms at M=1109
QSA (attention + scoring + pooling)	3.16	9.1%	`qsa_prefill_attn_g` 66–71 ms per layer-chunk
GDN (gates, conv, chunk recurrence, norms)	2.74	7.9%
BF16 GEMMs (`dense_gemm_bf16_pipelined`: GDN out_proj, HC down, PLE)	2.50	7.3%	GDN out_proj ~7 TF/s
HC (pre/post)	1.07	3.1%
dense attention, chunks 0–1 (`inferspark_prefill*`)	0.64	1.8%
shared expert	0.62	1.8%
other	0.68	2.0%

Not in the GPU sum:

7.44 s of host time right before the 60 qsa_block_pool launches. QSA ingest copies the raw-key column with copy_d2d_2d_async, and on every atlas_scale target that falls back to one cuMemcpyDtoDAsync per row: 2,048 per attention layer per chunk, 111,750 [d2d] ops in the trace. SYNC inflates this, so an unsynced trace is running now to price it.
The GDN qkvz BF16 GEMM. It goes through hipBLASLt (the serve log says cuBLASLt BF16 GEMM live … hipBLASLt shim), which the launch trace doesn't record.
3.7 s of host time before chunk 0's PLE embedding (first request after boot).

Root causes found in code:

Attention runs on FP8 copies on HIP. weight_loader/qwen35/load_layers/attention_arms.rs calls predequant_for_prefill unconditionally, so prefill takes the FP8 arms of cache_skip_qkv.rs / paged_qkv.rs / paged_oproj.rs. The GDN sibling (linear_attn_arms.rs:522-528) already skips this on atlas_hip, because HIP has no fast FP8 GEMM. The NVFP4 arm just below (w4a16_gemm_t_m128) ran the 12288×2560 q shape at 21.6 TF/s in strix job 146, about 6 ms. Today's FP8 arm takes 76 ms, or 149 ms in chunk 0, where activations are FP8-quantized too. Skipping the copies also frees ~0.6 GB.
Routed MoE: strix-hip: F16-operand prefill GEMM twins (exact NVFP4/FP8 dequant) for dense m128, routed MoE, w8a16 #155's F16 twins never engage (details on strix-hip: F16-operand prefill GEMM twins (exact NVFP4/FP8 dequant) for dense m128, routed MoE, w8a16 #155). Production runs the _t_k64 kernels.
Router: the untransposed w4a16_gemm gathers single bytes across 64 rows with two barriers per 16-K step, and its time jumps 9× between M=1109 and M=2048.
D2D: native HIP already links the cudart→HIP shim, which has cudaMemcpy2DAsync. The per-row loop is only needed for SCALE.

Plan (Devin): a HIP prefill train on winbox, built from local branches. Nothing gets pushed until the platform suite passes.

P0: main + qsa: O(complete) radix-select top-k on the device by default (Flash-Next decode past the 2051-token bound) #152 + lm_head: one-pass BF16 batch2 LM head for K=2 verify on HIP (Flash-Next, −5.9 ms/verify step) #153.
P2: P0 + the pitched D2D copy (byte-identical) + a bit-identical router GEMM twin (same dequant expression and WMMA chain, 64-K stages).
P4: P2 + attention on the NVFP4 arm + strix-hip: F16-operand prefill GEMM twins (exact NVFP4/FP8 dequant) for dense m128, routed MoE, w8a16 #155 + the MoE-twin wiring.

Measurements:

A/B: n=3 alternating at 10k / 20k / 29k.
Output checks: greedy byte-identity P2 vs P0, drift P4 vs P2.
Engagement: a traced run to confirm the new kernels actually execute.
Then: ST-995 and a cache-on replay on the winning tree.

Two checks on strix today: the router twin's bit identity (microbench) first, and the same trace-based attribution for the 27B (#72).

HIP prefill train: A/B result on winbox. Prefill TTFT is −47% to −51%. The byte-identical half is −10% on its own.

The four commits are local and unpushed (AMD rule) until the suite passes:

C1: pitched D2D copy on native HIP.
C3: bit-identical router GEMM twin w4a16_gemm_s64.
C2: attention prefill on the NVFP4 arm on HIP.
C4: strix-hip: F16-operand prefill GEMM twins (exact NVFP4/FP8 dequant) for dense m128, routed MoE, w8a16 #155's MoE F16 twins engaged in the k64 prefill arms.

Fingerprint: winbox plan p6, Flash-Next NVFP4, p4's PF recipe (util 0.90, seq 32768, prefill 2048, kv bf16, batch 1, MTP 1, PREFIX_CACHE=0). 3 rotated rounds, TTFT from the server's Done line, max_tokens 16. Trees: P0 11639c39 = main + #152 + #153; P2 4205250e = P0 + C1 + C3; P4 239b9a79 = P2 + C2 + #155 + C4.

prompt	P0	P2	P4
9,967 tok	42.8 s (42.8 / 42.8 / 43.1)	38.6 s, 0.901	20.8 s, 0.486
20,105 tok	85.0 s	76.1 s, 0.895	41.5 s, 0.488
29,237 tok	132.4 s	118.9 s, 0.898	70.0 s, 0.529

P2 vs P0: greedy output is byte-identical on 12/12 replies (4 prompts × 3 rounds).
P4 KV: 7,708 blocks (123k tokens) vs 6,061 (97k). That's +27% from dropping the FP8 attention copies.
P4 greedy: identical to P0 on the short and 29k prompts. On 8k/16k it changes in the expected way (C2/C4 change operand precision): deterministic across rounds, flipping between this prompt family's two answer forms. The ST-995 is running now.
Decode is unchanged.

Engagement, from a SYNC-traced P4 prefill. Same 9,302-token prompt as the original attribution:

Total: summed GPU 34.5 s → 14.6 s; span 51.1 s → 23.8 s.
D2D: [d2d] ops 111,750 → 126.

Per-launch times:

kernel	before	after
routed gate_up (`_t_k64` → `_t_f16a`)	33.0 ms	7.04 ms
routed down (`_t_k64` → `_t_f16a`)	18.4 ms	4.23 ms
router (`w4a16_gemm` → `w4a16_gemm_s64`)	17.4 ms	0.86 ms
attention q-proj (`fp8_*` → `w4a16_gemm_t_m128`)	75.7–149 ms	7.5 ms

No fp8_gemm_t_m128 / fp8_fp8_gemm_t_m128 launches remain.

Where P4's 14.6 s goes, i.e. the next levers:

QSA attention: 2.89 s (qsa_prefill_attn_g 61–65 ms per 2k chunk-layer; GB10's TC2 kernel has no gfx1151 port).
Routed-MoE twins: 2.70 s (~16–19 TF/s).
GDN chunk kernels: 2.65 s (ATLAS_GDN_PIPE is unmeasured on gfx1151).
BF16 dense_gemm_bf16_pipelined: 2.33 s. That's the GDN out_proj at 8.4 TF/s (7.6 ms for 64 GFLOP) plus HC down; it's the clearest remaining inefficiency.
HC: 1.08 s.

Chunk 0 also still pays ~4 s of host time on the first request after boot (PLE fault-in).

winbox memory was the blocker. Now unblocked (2026-10-09)

What was wrong

winbox's BIOS reserved only 32 GB for the iGPU, and Windows saw 95.8 GB as RAM. The GPU could borrow half of that through WDDM shared memory, so hipInfo reported ~77 GB.
Per the comment in crates/atlas-kernels/hip/libcuda_hip_shim.cpp, writes past roughly 63 GB faulted the context (hipErrorLaunchFailure).
Every Flash-Next launcher sets ATLAS_UMA_COMMIT_LIMIT_GB = "96" (scripts/strix-windows/win_serve_flashnext_nvfp4.ps1). When that variable is set, the shim reports it as total and free, so preflight logged GPU 0: 96.0 GB total, 96.0 GB free without measuring anything. That number was never real on this box.
Flash-Next needs 75.85 GB of on-disk weights, which preflight estimates as ~91 GB peak.

What it caused

Every Flash-Next serve on winbox since 2026-10-06 23:59 local died mid-load: 9 in a row, at shard 3–7/11, with no error line and never Server live.
- Logs: atlas-p5\out\serve-fnext-*.log, atlas-p5c\out\serve-fnext-{w2-*,p10,p10r3}.log.
- The last run that got up, serve-fnext-fnext-20261006-100144.log, ended in hipErrorLaunchFailure.
Earlier runs that did serve had no memory left for prefix caching. The 2026-09-30 replay spent 92% of its wall time re-prefilling (median TTFT 64.5 s), because a 16-slot Marconi pool would have pushed KV below 32k.

What changed

The BIOS iGPU allocation is now 96 GB dedicated (the maximum). Windows RAM is 31.8 GB.
hipInfo reports totalGlobalMem: 107.92 GB.
Weights fit with roughly 15 GB to spare, so prefix-cache slots are now affordable.

Before the next measurement

Re-measure the real commit ceiling under the new split. The ~63 GB fault point and the shim's 14 GB ATLAS_UMA_DRIVER_RESERVE_GB default were both measured under the old 32 GB carve-out.
Set ATLAS_UMA_COMMIT_LIMIT_GB in the launchers from that measurement, not a hand-picked 96 (it may now under-report).
Re-run the strix-windows: Flash-Next launchers + prefix caching on by default (stacked on #80) #149 prefix-cache replay with prefill: refuse mid-chunk tail capture on aux-carrying models (Flash-Next warm path never engaged on Windows) #157 applied, and compare against the 64.5 s median TTFT baseline.

Box notes:

winbox is headless-safe: Tailscale runs unattended and sshd is automatic.
Automatic Windows updates are now disabled by policy, so the box won't reboot unannounced mid-run.

NVFP4 prefill GEMM on gfx1151: one LDS buffer plus B-fragment reuse gives +15–28%, bit-identical. strix, isolated kernel bench.

Why. rocprofv3 on w4a16_gemm_t_m128_f16a (strix s40, 27B 4k prefill) reads:

5.35 waves per CU;
DRAM 46 GB/s;
MemUnitBusy 40%.

So it's occupancy-bound, not bandwidth-bound. Its double-buffered tiles take 41 KB of LDS, and only ~3 CTAs fit a WGP's 128 KB.

Change (branch perf/hip-w4a16-f16a-single-lds, 89a26b846 + d6057db77):

One LDS buffer: ~20.6 KB per CTA. The register prefetch still overlaps global loads with WMMA; the cost is a second barrier per K-step.
Each B fragment is read once per K-step for both 64-row halves.

VGPRs drop from 256 (+2 spilled, 12 B scratch) to 248 with no spill.

Measured (strix s45 / s45b): one process, all variants interleaved in rotating order, 15 rounds × 5 reps, medians. The output hash is identical to main on every shape.

shape (M×N×K)	main	single-LDS + reuse	Δ
27B gate/up 2048×17408×5120	21.9 TF/s	25.3	+15%
27B down 2048×5120×17408	21.1	24.3	+15%
27B attn 2048×6144×5120	20.2	25.7	+27%
FNext attn q 2048×12288×2560	19.2	22.8	+19%
FNext 2048×16384×2560	19.3	24.8	+28%
27B gate/up 512×17408×5120	18.3	22.5	+23%

What didn't work (recorded per the neutral rule):

B-fragment reuse alone is 16–25% slower: more live VGPRs at the same LDS occupancy.
Single buffer alone gains only +7–16%.
An F16-operand twin of the FP8 w8a16_gemm_n_m128 is more accurate (relL2 vs fp64: 2.3e-3 → 1.6e-3) but 5–15% slower. Not proposed.
Combining that FP8 twin with B-reuse ran ~6% slower end to end (s42).

Method note: separate-process GEMM benches on strix drifted 6–50% between runs (DPM clocks, auto level). Only in-process interleaved medians are usable for kernel A/Bs on this box.

Now running: end-to-end prefill A/B of test/prefill-2 against main. It contains this change plus the same single-LDS change for the routed-MoE F16 twins (perf/hip-moe-f16a-single-lds 9be597d00, 31 → ~15.6 KB). 27B on strix s46, Flash-Next on framework f61. Arms are interleaved ×3, and greedy and logprob probes must be identical.

Flash-Next NVFP4 prefill on gfx1151 Linux: profile, plus four bit-identical changes that didn't move it. Framework, main 8863dc5e5.

Where the time goes (f60, SYNC launch trace, cold 7,380-token prefill, 8.77 s summed GPU, 10.47 s wall):

kernel	share	note
`qsa_prefill_attn_g` (QSA attention, 12 layers)	22.6%	55 ms per launch, no LDS tiling (static analysis)
`moe_w4a16_fused_gate_up_t_f16a` + `grouped_gemm_ptrtable_t_f16a` (routed MoE)	26.8%	rocprofv3: 2.5 waves/CU, MemUnitBusy 15–19%
GDN chunk kernels (`delta_h_vfused`, `recompute_wu`, `fwd_o`)	17.6%
attention projections (`w4a16_gemm_t_m128`, `w4a16_gemm_t`)	11.2%	below the 40M N·K gate, so the BF16 base kernel runs
HC (`hc_pre_stage_bf16`, `hc_post`, `hc_pre_mix`)	~8%

End-to-end A/Bs, interleaved ×3 at ~1.9k / ~7.4k / ~12.5k. All arms identical on greedy and logprob probes unless noted.

change	result
f61: dense F16 GEMM single-LDS + B-reuse, MoE twins single-LDS (`test/prefill-2`)	neutral (−0.1 to −0.4%)
f61: `ATLAS_F16_M128_MIN_NK=0`, so attention projections use the F16 twin	+2–4%, but not identical (2k and 10k diverge); not pursued
f62: MoE twins stride their M tiles with the persistent short grid (`776320110`, the k64 kernels' fix ported)	neutral (±0.7%)
f63: + `ATLAS_MOE_PREFILL_EXACT_TILES=0` (no per-layer D2H offsets sync)	neutral; the 10k probe diverges (open question on that non-default path)

So for Flash-Next neither the MoE GEMM's occupancy nor the per-layer sync is what limits prefill. Steady state stays ~717 tok/s at 7.4k, against Gufo's 1,721 at 8k depth. Wall exceeds summed GPU time by ~1.5 s at 7.4k. f64 is an unsynced launch trace to attribute that host time.

MTP past the QSA bound is fixed by default. Branch perf/hip-qsa-verify-active-default (88cffac78) with ATLAS_QSA_VERIFY_ACTIVE unset (f58): MTP1 decode 18.75 tok/s at 8k and 17.64 at 32k cached depth, 0 "mtp declined: qsa_active" lines. On main the same rows decode at 12.1 tok/s. The branch is ready; the owner pushes it.

Flash-Next prefill is GPU-bound too: the "1.5 s host gap" was a measurement artifact (framework f65 + f64).

f65: rocprofv3 kernel + memcpy trace (no counters, unserialized) of a warmed 6,649-token prefill on test/prefill-7. Span 9.53 s, GPU busy 9.37 s, idle 158 ms (1.7%): 89 ms is one H2D copy pair at request start, and the rest is <50 µs launch gaps. The earlier "wall 10.25 s vs 8.77 s GPU" comparison set an unsynced wall against SYNC-mode per-kernel sums, which undercount. Neither the per-layer MoE offsets sync (f63 neutral) nor host launch overhead costs anything measurable.

f64 (unsynced launch trace) and the f60 copyBuffer storm: the 298k __amd_rocclr_copyBuffer (0.8 s) and moe_transpose_u8_batched in f60's rocprof are boot-time weight transposes. They appear 0 times in any request.

True kernel durations, warmed 6,649-token prefill:

kernel	share
`qsa_prefill_attn_g`	19.3% (scalar, ~2.2 TF/s)
`moe_w4a16_fused_gate_up_t_f16a` + `grouped_gemm_ptrtable_t_f16a`	24.6%
GDN chunk kernels	15.7%
hipBLASLt BF16 GEMMs (`Cijk_*`, GDN projections + HC)	~12%
`w4a16_gemm_t_m128` + `w4a16_gemm_t`	9.7%

Next lever (in flight): HIP never had a tensor-core QSA kernel. try_kernel("qsa_attn_tc2") returned 0 and prefill always ran scalar _g. Branch perf/hip-qsa-attn-tc2 881c0b550 adds a native HIP qsa_attn_tc2.cu. It's strix-hip's validated inferspark_prefill_paged_64 WMMA body with gb10 tc2's head tiling and gathered keys, and keeps the same entry name, arguments and grid, so ATLAS_QSA_ATTN_TC2=1 selects it with no Rust change. s65 (strix: relL2 + timing against _g) and f66 (framework: end-to-end A/B + probes) are queued. Not bit-identical, so the gate is ppl + needles, as for gb10 tc2.

HIP tensor-core QSA attention: −10% / −12% Flash-Next prefill at 7.4k / 12.5k (framework f66). Branch perf/hip-qsa-attn-tc2 881c0b550 (off main db442e2d4), one build, binary 4f10b99cb042. main arm = ATLAS_QSA_ATTN_TC2 unset (scalar qsa_prefill_attn_g, the same path as main); test = ATLAS_QSA_ATTN_TC2=1. Cold TTFT, interleaved ×3.

prompt	`_g` r1 / r2 / r3	tc2 r1 / r2 / r3	Δ
~1.9k	2.10 / 2.05 / 2.04 s	2.08 / 2.06 / 2.01 s	±0 (below the 2051-token QSA bound, kernel unused)
~7.4k	10.21 / 10.26 / 10.27 s	9.17 / 9.23 / 9.23 s	−10%, 719 → 800 tok/s
~12.5k	17.84 / 17.83 / 17.81 s	15.69 / 15.69 / 15.68 s	−12%, 704 → 801 tok/s

Faults: 0 in all six serves.

Numerics: the probes are identical below the bound (short / code / 2k; 2k mean |Δlogprob| 0.011). The 10k probe diverges at char 218 (mean |Δlogprob| 0.145). That's expected for any non-bit-identical change above the bound: QSA top-k selection is chaotic, and gb10's tc2 diverges the same way. As there, the gate is ppl + needles, not greedy identity. f67 is running:

scripts/ppl.py dense region (6k-char windows, must match) and above-bound (44k-char, ~11k-token windows);
12 needles (4k / 9k / 13k × 4 depths, thinking default).

The reference is gb10 tc2: dense 0.000%, above-bound −0.019%, needles 12/12. If it holds, gfx1151's default can flip (it's still opt-in pending exactly this measurement: qsa_attn_tc2_decision(..., gfx=true) → off).

HIP QSA tc2 passes gb10 tc2's gate (ppl + needles), framework f67. perf/hip-qsa-attn-tc2 881c0b550, binary 4f10b99cb042. Ref arm = ATLAS_QSA_ATTN_TC2 unset (scalar _g); test = =1. scripts/ppl.py over docs/.

check	`_g`	tc2
needles (4k / 9k / 13k × depth 0.1 / 0.35 / 0.65 / 0.9, thinking default)	12/12	12/12
ppl, dense region (6k-char windows, below the 2051 bound; 13,536 tokens)	9.6649	9.6649, max \|Δlp\| 0
ppl, above bound (44k-char windows, ~11k tokens; 80,002 tokens)	5.2421	5.2464, ratio 1.0008

ppl.py prints FAIL for the above-bound region. It's the per-token clause, not the ppl ratio (1.0008 against a 1.02 limit): one token moved 5.37 nats; mean |Δlp| is 0.090. Any non-bit-identical change past the bound trips this, because QSA top-k selection is chaotic. It's why gb10 tc2 was gated (and defaulted on) by ppl + needles rather than per-token or greedy identity. The engine is deterministic run-to-run, so this isn't noise; it's the expected selection drift. For comparison, gb10 tc2's above-bound ppl was −0.019%.

Default flip, as a separate commit for the owner to take or leave: 62e4d801d on the same branch makes qsa_attn_tc2_enabled() default ON for native HIP (cfg!(atlas_scale) && !cfg!(atlas_hip) as the "gfx" argument). Non-HIP SCALE builds stay off (unmeasured), and ATLAS_QSA_ATTN_TC2=0 restores scalar. f68 is checking that the unset-env path now runs tc2.

Correction to the f66 comment: the branch is based on main 87df79b2f (= db442e2d4 + #228, decode-only), not db442e2d4. Both A/B arms ran the same binary, so the numbers stand.

Navigation Menu

Repository navigation

[job] Qwen3.8-Flash-Next on Strix Halo: prefill and TTFT #144

Status (updated 2026-10-01 19:30Z)

winbox memory was the blocker. Now unblocked (2026-10-09)

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[job] Qwen3.8-Flash-Next on Strix Halo: prefill and TTFT #144

Description

Status (updated 2026-10-01 19:30Z)

Activity

AzeezIsh commented on Oct 1, 2026

AzeezIsh commented on Oct 2, 2026

AzeezIsh commented on Oct 2, 2026

AzeezIsh commented on Oct 4, 2026

AzeezIsh commented on Oct 4, 2026

AzeezIsh commented on Oct 9, 2026

winbox memory was the blocker. Now unblocked (2026-10-09)

AzeezIsh commented on Oct 28, 2026

AzeezIsh commented on Oct 28, 2026

AzeezIsh commented on Oct 28, 2026

AzeezIsh commented on Oct 28, 2026

AzeezIsh commented on Oct 28, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions