DSWS (MAD-305): dyn-VGPR fp8 GEMM — emergent wave economy, W=16 reliability, G_resident design v2 - #16
Merged
Conversation
…ign/plan Snapshot of the DSWS v2 claim-based split-K substrate and the Phase B runtime role-conversion design, after the static substrate passed its supervised GPU gate (all 3 role mixes x 2 split-K tiers, oracle bit-exact, zero bricks, 2026-07-01). Included (all new, isolated to spike/dvgpr_occ/): - occ_kernel_dsws.s: v2 claim-based split-K kernel (Phase A, GPU-green) - dsws_ctrl_model.cpp + test: control law (watermark/epoch/gate/reserve) - build_dsws.sh: offline assemble + RGA gate driver - SPEC/PLAN_DSWS_SUBSTRATE_V2.md: Phase A design + plan - SPEC/PLAN_DSWS_PHASEB_CONVERSION.md: Phase B conversion design + plan (per-epoch snapshot quiesce + bail-time commit; approved 2026-07-01) - SPEC/PLAN_DSWS_CONTROLLER.md, MAD305_DSWS_MASTER.md, RESULT_DSWS.md: context Deliberately excluded: pre-existing working-tree changes in occ_kernel_coop.s, occ_dispatch.cpp, fp8_oracle.*, mt_pagedattn* (entangled with unrelated run_mbcoop/CoopResult work) and build artifacts (*.bin/*.o/build/). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0132aDSBLwusCJ4KzHQTnvdu
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0132aDSBLwusCJ4KzHQTnvdu
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0132aDSBLwusCJ4KzHQTnvdu
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0132aDSBLwusCJ4KzHQTnvdu
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0132aDSBLwusCJ4KzHQTnvdu
Wire the conversion crux into occ_kernel_dsws.s (all under .if DSWS2_CONV,
byte-identical at DSWS2_CONV=0):
- conv_dec_floor: floor-guarded ds_cmpstore CAS decrement of a role slot
(never drives a role below 1 wave; two same-source converters/epoch safe).
- conv_apply: bail-time COMMIT — floor-dec source, reserve_try envelope,
dest inc, s59 role flip, s_alloc_vgpr GROW/SHRINK (SCC-retry), ordered
strictly BEFORE the QUIESCE_CNT bump (snapshot handshake, SPEC 3.4 dec 2).
- Per-role decision + commit at all 3 terminal bails (compute/afeed/bfeed),
dir a compile-time immediate per site.
- Quiesce sentinels switched to the epoch-parity snapshot (G+snap.nC etc.)
with a QUIESCE_CNT >= WAVES-1 cross-check; DIAG occ[29] mismatch flag.
- Broadcast snapshot of the live mix into the next epoch parity + QUIESCE
reset, epoch published last.
Offline gates green: 3/3 mixes ASSEMBLE_OK, RGA 0 spills; DSWS2_CONV=0
sha256 byte-identical; dry-print NCOMP=4/2/2 no REFUSE; CPU model ALL PASS.
Reviewed clean (spec + quality): OOR window safe at all 3 grow sites,
commit-before-QUIESCE ordering, floor-guard CAS, reserve_try undo verified.
rga_check.sh: make KSRC overridable (${KSRC:-default}) to gate the dsws
kernel; default unchanged. PLAN: fix a tautological inline test-assertion
example.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0132aDSBLwusCJ4KzHQTnvdu
…to-budget) Design spec for the DSWS Model-B wave-role economy, extending the Task 1-5 substrate. Covers: one s59-driven dispatcher unifying entry + re-dispatch (lands on _follow, scalar-only, zero new OOR); lean-start pool of N_POOL waves with a seed compute subset growing into real-SIMD-ceiling BUDGET headroom; no parking (pool sized so all waves stay active-lean, QUIESCE_CNT >= N_POOL-1 intact); cooldown K (default 0); DSWS2_FORCE deterministic bring-up hook; and the quiesce-reconciliation proof that re-dispatch makes the claimer's per-epoch snapshot equal the mix waves actually execute. Brainstormed 2026-07-01 (kmbandy). Awaiting writing-plans. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0132aDSBLwusCJ4KzHQTnvdu
…ction Add the 10-task implementation plan (offline T1-T6, supervised GPU gates T7-T10 each needing individual greenlight). Correct the spec (§1/§3/§6/§9): every wave already launches lean-32 (compute grows to NFV per-rowblk, not at init), so seeding a role is s59-assignment only — no launch-time s_alloc_vgpr, no new OOR grow site. Drop N_POOL/SEED_* (pool size == launched mix sum, seed == the existing wid partition); conv_apply's grow remains the only GROW in the design. BUDGET retune knob + a compile-time WAVES*VLEAN<=BUDGET no-parking invariant carry the grow-into-budget economy within the launched mix. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0132aDSBLwusCJ4KzHQTnvdu
…CKED, needs offline root-cause Offline pool-economy implementation, all review-clean, all offline gates green (assemble matrix, RGA 0-spill, CONV=0 .text byte-identical to Phase A, CPU model, dry-print): - T1 CPU model: role_dispatch/cooldown_step/pool_fits_lean/quiesce_ready_pool - T2 seed s59 in the lean partition arms + BUDGET headroom knob (no launch grow) - T3 .Ldispatch trampoline unifying entry + re-dispatch (targets _follow, scalar-only) - T4 cooldown K on s66 (default 0; prologue-dead scalar, liveness verified) - T5 DSWS2_FORCE deterministic bring-up hook (all 4 dirs assemble) - T6 offline integration gate: green KNOWN BUG (found on first-ever CONV=1 silicon dispatch, Pool-T7 inert-safe gate): CONV=1 with conversions PROVABLY DORMANT (CTRL_LOW=0/HIGH_A=6/HIGH_B=4, every watermark branch unreachable) HUNG the GPU -> MODE1 reset. Phase-A CONV=0 ran clean across all mixes, so the hang is in the always-active Phase-B machinery (the .Ldispatch trampoline and/or the snapshot/quiesce handshake), NOT the conversion logic. Prime suspect: the claimer's snapshot-sized quiesce sentinels spin forever. Root-cause is OFFLINE-ONLY from here (extend the CPU control model to simulate the multi-wave quiesce/dispatch handshake); no further gfx1201 dispatch until both the deadlock is understood AND the compositor is isolated from the card. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0132aDSBLwusCJ4KzHQTnvdu
…brick) The CONV=1 seed block sent first-time wave entry through the .Ldispatch re-dispatch trampoline, which lands on _follow and deliberately skips _alloc/_init. That is correct for RE-dispatch (a wave that already ran them once after a bail) but fatal for first entry: the three per-wave initializers live ONLY inside _alloc/_init -- (a) s_alloc_vgpr 32 -- the DYNVGPR per-wave lean allocator handshake, (b) INITFLAG==0xACED -- the LDS rendezvous with the claimer, (c) s35 = 0 -- the local last-seen-epoch baseline. First-entry-via-_follow skipped all three, so followers read EPOCH/STI from uninitialized LDS with a garbage s35 before the claimer published INITFLAG, desynced from the epoch clock, never reached their _quiesce bail, and QUIESCE_CNT never reached WAVES-1 -> the claimer spun forever in .Lclaimer_wait_done -> GPU hang -> MODE1 reset -> desktop brick. This is why Phase-A CONV=0 ran clean (its .else entry goes .Lbfeed/.Lafeed/.Lcompute -> full _alloc/_init/_follow) but the first CONV=1 dispatch hung with conversions provably dormant: seed->dispatch entry routing was the only new always-active path. Entry and re-dispatch are NOT interchangeable. Fix (CONV=1-only): the 3 seed arms now s_branch .Lcompute/.Lafeed/.Lbfeed (full role entry, s59 pre-seeded) instead of .Ldispatch. Bails still go _quiesce -> .Ldispatch -> _follow (re-dispatch unchanged). Corrected the .Ldispatch header comment to RE-DISPATCH ONLY (and fixed its inaccurate "already-consumed INITFLAG" note -- INITFLAG is written once and never cleared; the real reason re-dispatch skips _init is it would reset s35). CPU model (dsws_ctrl_model.cpp): added the first-entry contract (WaveEntry/simulate_first_entry/entry_safe/claimer_quiesce_converges) + a Pool-T7 repro test (TDD RED->GREEN): the buggy routing deterministically fails to converge (reproduces the hang), the fix converges. Offline validation: CONV=0 .text sha256 byte-identical to HEAD on all 3 mixes (4c2a2b/6c1a1b/2c3a3b); CONV=1 .text differs as intended; RGA 0-spill, max-live VGPR 83 <= NFV=112; CPU model ALL PASS. Independently Codex-reviewed offline (no blocker/major). GPU re-baseline pending kmbandy greenlight. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0132aDSBLwusCJ4KzHQTnvdu
Propagate the Pool-T7 root cause (fixed in b01c722) back into the design docs so the "unified dispatch" language can't reintroduce it. - Spec §2: `.Ldispatch` is RE-DISPATCH ONLY, not a unified entry+re-dispatch mechanism. First entry runs the full role `_alloc`/`_init`/`_follow`; only a post-bail wave lands on `_follow`. Also corrected the inaccurate "INITFLAG already consumed" rationale (INITFLAG is written once, never cleared; the real reason re-dispatch skips `_init` is it would reset `s35`). - Spec §3: seed arms fall into the full role entry, NOT `.Ldispatch`. - Plan architecture summary + Task 3: Step 3 no longer flips the seed arms to `.Ldispatch` (that was the exact instruction that caused the brick); the arms stay on `.Lcompute`/`.Lafeed`/`.Lbfeed`. Task 3 retitled/re-scoped to re-dispatch only; commit-message template updated. No code change (docs only). Fix itself is b01c722. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0132aDSBLwusCJ4KzHQTnvdu
…tion + race harness Root-caused the residual CONV=1-dormant hang (post entry-fix b01c722) to the KNOWN multi-grower dyn-VGPR collision (ISA §3.3.3.2): occ_kernel_dsws.s .Lcompute_grow is a bare s_alloc_vgpr SCC-retry with no sum-envelope and no stagger, so all NCOMP compute waves grow to NFV=112 in lockstep and collide. Confirmed by: offline std::thread QUIESCE/epoch model (0/400 stalls) + Codex static review ruling OUT the protocol logic; and a heisenbug — adding DIAG wedge-frame instrumentation shifted timing and the CONV=1/DIAG=1 4c2a2b bin ran CLEAN (ok=1536 bad=0, all chunks, dmesg silent) where the stock bin hung. kmbandy confirmed the collision and the fix direction. Contents: - HANDOFF_DSWS_ROLLING_DYNVGPR.md — architecture (NOT yet detailed spec) for the fix: one pool + Σ≤BUDGET sum-envelope, real split-K (n_kseg>1) for per-kseg grow→flush-to-C→immediate shrink, every compute grow routed through the shared vgpr_reserved envelope (collision unreachable by construction), stagger for compute-saturation + collision-avoidance + feed fungibility, instrumentation baked in. Reconciles with SPEC_WAVESPEC traveling-peak (KG 1fdd5784), MAD305_DSWS_MASTER split-K framing, and the vgpr_reserved envelope. Full debug arc, references, safety rules, next-session tasks. - occ_kernel_dsws.s — wedge-frame DIAG instrumentation (epoch_mark macro + claimer advance-gate counter snapshot into host-streamed occ slots). CONV=0 and CONV=1/DIAG=0 .text byte-IDENTICAL to HEAD; markers only at CONV=1 && DIAG=1. RGA 0-spill, max-live 84. - test_dsws_quiesce_race.cpp — offline std::thread claimer+followers QUIESCE/epoch race harness (400 trials × {seq_cst, relaxed}, 0 stalls — ruled out the protocol race). Not-ours siblings (occ_kernel_coop.s, occ_dispatch.cpp, fp8_oracle.*) left untouched/unstaged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0132aDSBLwusCJ4KzHQTnvdu
…racy-ti OOB brick The SAFEPROBE clamp only pinned the per-lane vaddr regs (v8/v9/v10); its own comment (line 752) noted it "pairs with the future ti clamp" — which was never implemented. That was the hole: a racy/torn `sti` read during the claimer's per-super-tile republish decodes a garbage `t` -> garbage `mblk`/`tcol` -> the A/B/C SCALAR base goes out of buffer -> gfxhub page fault -> MODE1 brick (the COOP_STATUS.md:145 racy-garbage-ti->OOB class). Fix: clamp t = min(sti>>shift, TOTAL-1) inside DECODE_STI, under SAFEPROBE, using s11=TOTAL (userdata, never clobbered) and s36 (existing DECODE_STI scratch). With t/mblk/tcol now bounded — alongside the already-bounded ksi (mask), r/f (claim checks) and v8/v9/v10 (existing vaddr clamps) — EVERY global address a wave computes is provably in-buffer, so the OOB is unreachable by construction. The clamp is a no-op for valid indices. Result: CONV=0 stays green (ok=1536 bad=0). CONV=1 (4c2a2b, n_kseg=1, DIAG=0) — the exact config that bricked 3x today — now completes clean and CORRECT across 7/7 dispatches (ok=1536 bad=0 max_rel=0, dmesg silent). The apparent "hang" was MES-stuck after the page fault, not a separate liveness bug. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0132aDSBLwusCJ4KzHQTnvdu
…folding Checkpoint of the SUSPECT #2 debugging session (root cause + fix landed separately in the DECODE_STI ti clamp, f013114). Everything here is either offline-only or gated OFF by default and byte-identical when disabled — the CONV=1/DIAG=0 .text is unchanged (490db6e5) with all new symbols at their defaults. occ_kernel_dsws.s (gated scaffolding, all default-off): - DSWS2_ENVELOPE: the rolling dyn-VGPR sum-envelope (per-rowblk reserve/grow/ flush/shrink/release via vgpr_reserved) + PEAK_CONC; reserve_try/BUDGET gate widened to DSWS2_CONV||DSWS2_ENVELOPE so the envelope can run at CONV=0. Isolation-tested green at CONV=0 (the perf lever for the next phase). - DSWS2_GQUIESCE: device-scoped global QUIESCE handshake (a candidate fix that the ti clamp obviated; kept for reference, shelved). - DSWS2_BAILMARK: per-wave bail-epoch localization marks (diagnostic; host readout lives in the still-WIP occ_dispatch.cpp, not committed here). Offline models/tests: - dsws_ctrl_model.cpp + test_dsws_ctrl_model.cpp: reserve_spin/reserve_release + envelope-invariant test. - test_dsws_envelope_race.cpp: forward-progress thread race for the burst reserve (0 stalls, all PEAK_CONC>=1). - test_dsws_straggler.cpp: liveness/straggler model that showed the CONV=1 hang needed a hard-stuck follower, ruling out the ordering hypothesis. Docs: the rolling dyn-VGPR envelope design spec + implementation plan. build_dsws.sh: mk2 passthrough for the new envelope defsyms (default-off). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0132aDSBLwusCJ4KzHQTnvdu
… diag Snapshot the DSWS flow-economy work in progress (MAD-305, gfx1201/RDNA4): - occ_kernel_dsws_flow.s: emergent wave economy (baked NCOMP/NAFEED/NBFEED mix deleted; compute-biased seed; host-derived W_launch; VBUDGET physical budget) + RETBARRIER count-to-WAVES collective exit (all waves check in at .Lflow_dead and s_endpgm together so the EOP registers a clean completion; fixes the W=16 fence-never-fires stall: single-shot 25s-hang -> 0.81s clean). - occ_dispatch.cpp: run_dsws2 flow path (Wlaunch derivation, kernel-done gate, per-chunk EOP-on-last, periodic compositor yield) + ML8_CHUNK_DIAG per-chunk wall + STAGINSTR delta diagnostic (host-only, opt-in). - build_flow.sh: WAVES/VBUDGET/G/TRACE defsyms for the flow bin. - Supporting: coop/dsws/ring/grind kernels, oracle, and design/plan docs. WIP on feat/dsws-phaseb-conversion; open item = W=16 re-launched-chunk 2s straggler (compositor VGPR-contention hypothesis, under diagnosis). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0132aDSBLwusCJ4KzHQTnvdu
Root cause of the W=16 chunked-dispatch brick + wall instability (measured 2026-07-05): deadman_check does an s_sendmsg_rtn RTC read (an SQ-front-end op) at every loop-head. With 24 WGs x 16 waves = 384 waves and only 2 tiles/chunk, ~382 idle waves spin the coast loop and hammer that message bus ~41k times per chunk. That front-end spam (a) destabilized the coast wall -- identical STAGINSTR work measured 0.324s vs 1.996s across two chunks -- and (b) starved the compositor's SQC(inst) instruction fetch -> GCVM_L2 page fault -> MODE1. Fix: throttle the RTC read to once every DEADMAN_EVERY (=64) iterations via a per-wave counter in s71 (the high-RTC reg, unused at TRACE=0; deadman_check only reads s70). Force-retire slack = DEADMAN_EVERY iters (negligible vs 0.5s deadline). Result (12-chunk W=16, same config as the bricking run): all chunks complete, wall flat at 0.325s (6x variance gone), oracle CLEAN bad=0 max_rel=0, no brick. grow-fail=0 throughout also disproves the VGPR-starvation-livelock hypothesis. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0132aDSBLwusCJ4KzHQTnvdu
… '=' file The rdna4_fp8_gemm/dvgpr_occ spike accumulated local-only clutter (build dirs, compiled sim/probe binaries, .err/.co/.amdisa dumps, benchmark result logs, root-level results.db/json, a generated phase_profile.html, and scratch sources). Add scoped .gitignore rules so `git status` stays clean and these never leak into commits. Patterns are anchored per-path and use explicit prefixes so tracked CMakeLists.txt / TRITON_PIN.txt / *_DESIGN.md are untouched. Also removes a 0-byte file literally named '=' left by a redirect typo. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0132aDSBLwusCJ4KzHQTnvdu
…er (gated off) Step-2 design doc rewritten to v2, folding the council (Fable) review of v1: - H1 (silent reduction race) fixed: zero-init banks + all-ds_add => order- independent merge, the precondition for J>1 and POOL_N>1. - H3 (LDS arithmetic) fixed: real +256 header; measured OPSTRIDE 8192@SEGK32, 4096@SEGK16 from Run-4 LDS=57600; both feasibility tables corrected. - H5 (group/tile boundary bank collision) dodged: Path A keeps whole-tile groups (G_resident=G); Path B (partition) demoted to an optional post-bind throughput lever with drain-before-admit spelled out. - Reframes concurrent-fat as count x duty-cycle, with burst length J as the free (no-LDS) knob on the duty axis; adds the Gate-3 honesty check (does TF actually move with J, or is the cap structural?) and the occ[58] FATMAX instrumentation TODO. Kernel/build changes are the WOFLUSH (atomic-flush-to-C) experiment lever, flag-gated OFF: WOFLUSH=0 is byte-identical to the shipped throttle bin. Kept in-tree as a rejected-but-documented alternative to the on-chip reduction. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0132aDSBLwusCJ4KzHQTnvdu
Snapshot of local work-in-progress on the paged-attention path (mt_pagedattn.cu, mt_pagedattn_aiter.cu, mt_pagedattn_turbo_fp8.cuh) so it rides along in this branch push. Small, self-contained (14 insertions / 14 deletions); unrelated to the DSWS spike work in the rest of the branch. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0132aDSBLwusCJ4KzHQTnvdu
github-actions
Bot
added
documentation
…nversion # Conflicts: # ggml/src/ggml-cuda/mt_pagedattn.cu
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
DSWS (MAD-305) — dyn-VGPR fp8 GEMM spike for RDNA4 / gfx1201 (R9700, wave32, raw PM4)
Feature branch snapshot of the Dynamic-Split/Staggered Wave-Specialization kernel work.
Experimental spike under
ggml/src/ggml-cuda/aiter-integration/rdna4_fp8_gemm/spike/dvgpr_occ/—not intended for upstream; opened against
origin/master(fork) to track the branch.Highlights since
origin/mastercompute-biased seed + host-derived
W_launch+ a physical VGPR budget.s_endpgmso EOP registers cleancompletion.
deadman_checkspamming the realtimemessage bus from idle coast waves (starves the compositor's instruction fetch → page fault →
MODE1). Fixed by throttling the RTC read (
DEADMAN_EVERY=64): 12-chunk W=16 clean,bad=0,flat ~0.325 s/chunk, no brick.
DSWS_GRESIDENT_DESIGN.md) — burst-owns-a-ksi-run. Folds a councilreview: fixes a silent reduction race (zero-init banks + all-
ds_add), corrects the LDSarithmetic, dodges the group-boundary collision (whole-tile groups), and reframes concurrent-fat
as count × duty-cycle with burst length
Jas a free (no-LDS) knob. Design-only; not yet built.Notes for review
occ_kernel_dsws_flow.sis flag-gated OFF(
WOFLUSH=0is byte-identical to the shipped bin) — a documented, rejected alternative.mt_pagedattn*, last commit) so itrides along with the branch; isolated in its own commit.
.gitignorerules for the spike's local build artifacts / scratch / result logs.🤖 Generated with Claude Code