diff --git a/docs/current-state.md b/docs/current-state.md index ee69acd3..5dda3e52 100644 --- a/docs/current-state.md +++ b/docs/current-state.md @@ -49,6 +49,21 @@ See [`roadmap.md`](roadmap.md) and for the accepted forward contract. No M29 implementation or HIP-VMM investigation has begun. +### External gfx906 reference discovery — 2026-10-02 + +The current roadmap now includes a bounded M29.0A calibration against +`kyuz0/mi50-gfx906-toolboxes@a708a2790fa51303e3d4f5af9e53c045177181a3`. +That project provides a reproducible gfx906 ROCm 7.2.1 environment, a native +llama.cpp build, rebuilt gfx906 rocBLAS/Tensile/RCCL support, and patched +vLLM/Triton/FlashAttention paths. This is planning evidence only: MIInfer has +not adopted those libraries or frameworks and has not changed its runtime. + +The first authorized use is to refresh the external llama.cpp control with +MIInfer's exact model and qualified hardware/measurement contract. A later +OCI/Podman packaging path may reuse the environment/provenance lessons without +making generic frameworks part of the core runtime. See +[`gfx906-toolbox-reference.md`](gfx906-toolbox-reference.md). + ## Historical checkpoint — M28 / V2-0043 (2026-09-29) Everything after this historical archive banner is retained project history. diff --git a/docs/gfx906-toolbox-reference.md b/docs/gfx906-toolbox-reference.md new file mode 100644 index 00000000..52f813f9 --- /dev/null +++ b/docs/gfx906-toolbox-reference.md @@ -0,0 +1,123 @@ +# gfx906 Toolbox Reference and MIInfer Follow-up + +## Status + +Roadmap input only. No MIInfer runtime or dependency change has been made. + +## External source pin + +- Repository: `kyuz0/mi50-gfx906-toolboxes` +- Pin: `a708a2790fa51303e3d4f5af9e53c045177181a3` +- Relevant path: ROCm 7.2.1 `llama.cpp` toolbox +- Discovery date: 2026-10-02 + +The source pin is intentionally exact. Later refreshes require a new recorded +pin rather than silently following the external repository. + +## What the toolbox contributes + +The project is an enablement and reproducibility stack rather than an inference +engine competing directly with MIInfer. Relevant mechanisms include: + +1. A gfx906-native llama.cpp build with HIP enabled and + `GGML_HIP_ROCWMMA_FATTN=ON`. +2. Rebuilt ROCm libraries for gfx906, including rocBLAS/Tensile and RCCL, plus + extracted gfx906 Tensile artifacts for modern ROCm releases. +3. A patched gfx906 vLLM stack using gfx906-specific Triton and FlashAttention + forks. +4. A vLLM LDS compatibility correction that reduces a V1 prefix-prefill + `BLOCK_M` from 128 to 64 to stay within gfx906's 64 KiB LDS limit. +5. Container/toolbox packaging that makes the environment substantially easier + for another MI50 owner to reproduce. + +These are external implementation facts, not MIInfer adoption decisions. + +## Candidate A — M29.0A external-reference calibration + +### Question + +Has the strongest practical upstream-class gfx906 baseline moved materially +since V2-0045, and if so, why? + +### Contract + +Use: + +- 1 x AMD Instinct MI50 32 GB, gfx906/Wave64 +- 225 W cap +- 1606 MHz SCLK / 1000 MHz MCLK +- exact MIInfer `Qwen3.8-27B-Q4_K_M` model artifact and SHA-256 + `7e78da5d7e3ae28d178121f58646953305f3e5bd3cb46f4a75584e8b6c6fe169` +- deterministic equivalent prompt/decode token streams +- V2-0045 comparable points: P512, P1024, P2048, P4096, P8192 and TG128 +- at least five valid samples per comparable cell +- clocks, temperature, power, VRAM, process ownership, build/source provenance, + and raw timing retained + +Capture the exact llama.cpp submodule revision resolved by the toolbox pin and +the ROCm/HIP compiler, rocBLAS/Tensile provenance, and build flags. + +### Stop / attribution gate + +- If equivalent medians stay within 3% of the pinned V2-0045 upstream control + and no qualitative capability changes, record the refreshed baseline and stop. +- If a cell moves by more than 3% reproducibly, isolate source revision, + ROCm/Tensile, ROCWMMA attention, compiler/build flags, or another measured + cause before changing MIInfer. +- Do not infer a MIInfer optimization from an external aggregate result. +- Do not compare the toolbox vLLM 500-request/512-output-token throughput run + directly with MIInfer's single-stream TG128 model-forward measurement. + +## Candidate B — gfx906 reference environment + +If Candidate A demonstrates that the toolbox environment is stable and useful, +retain a provenance-pinned external reference environment for future M31 +requalification. + +The purpose is reproducibility, not framework adoption. MIInfer may use the +environment to build or run comparison implementations and to archive gfx906 +userspace components that disappear from newer ROCm distributions. + +Rebuilt rocBLAS/Tensile/RCCL artifacts should remain outside the core MIInfer +dependency set unless a controlled experiment demonstrates a direct need or +benefit for MIInfer. + +## Candidate C — MIInfer OCI/Podman distribution + +After the reference environment is qualified, build a minimal reproducible +MIInfer image/toolbox that: + +- pins the required ROCm/gfx906 userspace and compiler/runtime provenance; +- contains the released MIInfer executable and required runtime libraries; +- exposes `miinfer doctor` or equivalent hardware/runtime validation; +- validates gfx906, available VRAM, ROCm/HIP compatibility, and model hash; +- runs a bounded `miinfer serve` health/generation smoke; +- does not pull PyTorch, vLLM, Triton, FlashAttention, or llama.cpp into the + production execution path unless separately authorized. + +This is productization/reproducibility work. It must not change the custom +single-model execution architecture merely to match the toolbox layout. + +## Explicit non-goals + +This discovery does not authorize: + +- replacing MIInfer with llama.cpp or vLLM; +- adding continuous batching to M29; +- adopting Triton as a production dependency; +- porting the external FlashAttention kernel without profiling evidence; +- copying the `BLOCK_M=64` workaround into MIInfer without an exact-shape + experiment; +- starting a new unbounded kernel optimization campaign; +- changing M29/M30/M31/V3 ordering. + +## Roadmap placement + +- **M29.0A:** bounded current external-reference calibration. +- **M31:** reuse the refreshed reference as part of final single-MI50 frontier + qualification. +- **Cross-cutting release/productization:** OCI/Podman reproducibility path once + the reference environment is understood. + +The external project is therefore useful as a control and packaging blueprint, +not as a replacement architecture. diff --git a/docs/post-m26-performance-roadmap.md b/docs/post-m26-performance-roadmap.md index f654d97d..f071e299 100644 --- a/docs/post-m26-performance-roadmap.md +++ b/docs/post-m26-performance-roadmap.md @@ -297,12 +297,23 @@ class; the M29 two-shard/one-MI50 experiment is not dual-GPU performance evidence. See the canonical roadmap and D032 for ownership, identity, N=1/N=2, and the >1% decode-regression gate. -Planned stages are M29.0 long-context baseline (16K/32K/64K/128K), M29.1 -backing feasibility, M29.2 ContextSpace/logical pages, M29.3 DeviceKvPool and -DeviceKvShard, M29.4 two logical shards on one MI50, M29.5 optimized physical -view/attention integration, and M29.6 production 128K qualification. Each -stage requires its own focused goal; this document authorizes no -implementation. +Planned stages are M29.0 long-context baseline (16K/32K/64K/128K), M29.0A +current gfx906 external-reference calibration, M29.1 backing feasibility, M29.2 +ContextSpace/logical pages, M29.3 DeviceKvPool and DeviceKvShard, M29.4 two +logical shards on one MI50, M29.5 optimized physical view/attention integration, +and M29.6 production 128K qualification. Each stage requires its own focused +goal; this document authorizes no implementation. + +M29.0A is deliberately bounded. Re-run the exact MIInfer Qwen3.8-27B-Q4_K_M +artifact against the ROCm 7.2.1 llama.cpp path from +`kyuz0/mi50-gfx906-toolboxes@a708a279` under the same qualified MI50 power, +clock, token-input, and measurement contract used by V2-0045. If equivalent +cells remain within 3% of the pinned upstream medians, preserve the refreshed +environment as a reproducibility control and stop. A reproducible >3% movement +requires attribution before MIInfer adopts any environment, library, kernel, or +runtime change. The toolbox's patched vLLM/Triton/FlashAttention stack is a +separate throughput/compatibility reference, not a direct single-stream decode +oracle. See [`gfx906-toolbox-reference.md`](gfx906-toolbox-reference.md). --- diff --git a/docs/roadmap.md b/docs/roadmap.md index 0262cd97..993892b6 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -81,6 +81,7 @@ Stages (each requires its own scoped goal and gate): ```text M29.0 — Current long-context baseline: 16K / 32K / 64K / 128K +M29.0A — Current gfx906 external-reference calibration M29.1 — Physical-backing feasibility: HIP-VMM investigation M29.2 — ContextSpace + logical pages M29.3 — DeviceKvPool + DeviceKvShard @@ -89,6 +90,40 @@ M29.5 — Optimized physical KV view / attention integration M29.6 — Production 128K qualification ``` +### M29.0A — Current gfx906 external-reference calibration + +Before M29 changes context allocation or backing, refresh the strongest practical +external gfx906 baseline under the same MI50 conditions. The initial reference is +`kyuz0/mi50-gfx906-toolboxes` pinned at +`a708a2790fa51303e3d4f5af9e53c045177181a3`, using its ROCm 7.2.1 +`llama.cpp` toolbox path. This is a bounded calibration lane, not a new +optimization campaign. + +The first screen must use MIInfer's exact +`Qwen3.8-27B-Q4_K_M` model artifact/hash, the qualified 225 W and +1606/1000 MHz hardware policy, deterministic equivalent token inputs, and the +existing V2-0045 comparison points (P512/P1024/P2048/P4096/P8192 plus TG128) +where the external runtime can execute the same work. Record the toolbox commit, +llama.cpp submodule revision, ROCm/HIP compiler identity, gfx906 rocBLAS/Tensile +provenance, build flags, clocks, temperatures, power, VRAM, and raw samples. + +Use the result only as an attribution gate: + +- if the current toolbox reference stays within 3% of the pinned V2-0045 + upstream medians across equivalent cells and exposes no qualitative change, + record the environment and stop; +- if an equivalent cell moves by more than 3% reproducibly, attribute the + difference to source revision, ROCm/Tensile, ROCWMMA attention, build flags, + or another measured cause before changing MIInfer; +- do not compare vLLM aggregate multi-request throughput directly with + MIInfer's single-stream TG measurements; +- patched Triton, FlashAttention, vLLM, rocBLAS, RCCL, or Tensile components do + not become MIInfer production dependencies merely because the toolbox ships + them. + +The detailed discovery and bounded implementation candidates are recorded in +[`gfx906-toolbox-reference.md`](gfx906-toolbox-reference.md). + Single-MI50 production qualification targets 128K. The logical architecture may be designed for 256K capacity where inexpensive and technically useful; this does not require 256K physical backing on one card. The later V3 @@ -186,6 +221,25 @@ question remains whether MIInfer's architecture generalizes beyond the initial model without becoming generic, but the nearer question is how far MIInfer can push its intended agent workload on the targeted MI50 hardware. +## Cross-cutting — gfx906 reproducibility and distribution + +The toolbox discovery also exposes a productization opportunity that is +independent of kernel research: make the qualified MIInfer runtime reproducible +for another MI50 owner without requiring reconstruction of the development +environment. + +After M29.0A establishes the external reference environment, MIInfer may add a +provenance-pinned OCI/Podman distribution path for gfx906. It should package +only the userspace/runtime pieces MIInfer actually needs, preserve the exact +ROCm/toolchain and model/runtime provenance, expose hardware/environment checks, +and run a bounded `miinfer serve` smoke. Rebuilt gfx906 rocBLAS/Tensile/RCCL +artifacts remain optional reference or compatibility inputs unless controlled +evidence shows that MIInfer requires them. + +This is packaging/reproducibility work, not authorization to replace MIInfer's +custom execution path with llama.cpp, vLLM, Triton, or a generic framework. It +does not change the M29 → M30 → M31 → V3 milestone order. + # Roadmap Principles ## Evidence before architecture