Repository navigation
spark-model: refuse ATLAS_QSA_ATTN_L8 on gfx1151 + scripts/ppl.py gate - #167
Conversation
The QSA prefill-attention variants (ATLAS_QSA_ATTN_L8, _TC, _SCORE_TC) are marked NOT bit-identical and gated pending a perplexity check; this is the scripts/ppl.py those comments point at. Uses the legacy /v1/completions loglikelihood call (echo=true, logprobs=1, max_tokens=0), stdlib only. Verified against a live strix-hip serve (nvidia/Qwen3.8-27B-NVFP4): 794-token window of docs/, ppl=5.82.
…n gfx1151) The qsa_indexer.cu shared into strix-hip by symlink carries qsa_prefill_attn_l8, and on gfx1151 it destroys the context mid-prefill (winbox plan1006p9: ~18k-token echo/logprobs request, serve died, teardown swept 229,520 allocs, cuMemFree_v2 status 719; the default kernel scored all four windows clean, ppl=4.5917 over 73,591 tokens). This is a hard fault, not the numeric-divergence question scripts/ppl.py was meant to answer, so the env opt-in is refused outright on atlas_scale (with a one-line warn), via a new pure qsa_attn_l8_selectable mirroring qsa_attn_tc2_decision + table test.
|
gb10 counterpart check: Net: the refusal here is gfx1151-scoped only — on GB10 the opt-in still works (Devin) |
|
Flash-Next on reiner (gb10, postmerge
VERDICT: FAIL — a token moved >0.5 nats. The (Devin — job 744-gb10-l8-ppl-r2) |
Summary
Two pieces from the
ATLAS_QSA_ATTN_L8gating work:spark-model: refuseATLAS_QSA_ATTN_L8onatlas_scale— theqsa_prefill_attn_l8kernel (shared into strix-hip'sqsa_indexer.cuby symlink from the GB10 target) destroys the GPU context mid-prefill on gfx1151, not merely diverges numerically. New pureqsa_attn_l8_selectablemirrorsqsa_attn_tc2_decision, plus a one-line warn when the env opt-in is refused. NVIDIA selection is unchanged.scripts/ppl.py— the gate the_l8/tc/tc2kernel comments point at: legacy/v1/completionsecho+logprobs+max_tokens=0loglikelihood scorer (stdlib-only), plus--comparefor per-token Δlp and ppl ratio between two captures.Evidence
plan1006p9, same build both arms (atlas-p5@35b4db9bc): REF (default QSA) scored all 4 × ~18k-token windows — ppl=4.5917 over 73,591 tokens, 0 faults.L8ONserve died mid first window; teardown swept 229,520 allocations andcuMemFree_v2→status 719. Reported on [job] Qwen3.8-Flash-Next: long-context cold prefill to SGLang parity (stage 1 = #68) #74.cargo test -p spark-model qsa_attn):qsa_attn_l8_selectable_table+qsa_attn_tc2_decision_tablegreen.ppl.pyverified live on strix-27B (ppl=5.82 docs window) and winbox Flash-Next (4.5917).Related: #74 (QSA campaign), #163 comment for the SSM/prefix battery attribution.
🤖 Generated with Devin