docs: mark ROCm as supported and refresh the GB10 benchmark record

CHANGELOG.md

@@ -6,6 +6,10 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 
 ## [Unreleased]
 
+### Changed
+
+- AMD ROCm is documented as an officially supported backend since 0.8, on Linux and on Windows x86_64, rather than experimental. It remains a source build, and the validated configuration and the open gaps are unchanged; see the platform matrix and the Linux and Windows ROCm sections in [Installation](docs/installation.md).
+
 ## [v0.8.1] - 2026-11-03
 
 ### Added
@@ -73,7 +77,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 - CUDA server prompt lookup pauses rows whose recent verify rounds emit fewer than four tokens, resumes after seven shadow tokens confirm copying, and re-primes proposal-free ticks sooner (#2264). An isolated zero-accept miss after strong copy evidence receives one round of grace; partial low-yield matches do not. The CLI governor and non-CUDA governor policy are unchanged. The paged context cap is 4608 total tokens, allowing the existing 4096-target copy workload and its 256-token continuation while retaining the long-context guard. The final quiet GB10 measurement passes every criterion: chat −0.275%, short-copy +149–159%, and long-copy 1.52–2.82×.
 
 - `mlxcel serve` and `mlxcel-server` accept `--draft-kind prompt-lookup` with no drafter checkpoint: each eligible request runs prompt-lookup verify rounds inside the batch scheduler, at most one per tick, while proposal-free ticks stay on the lookahead pipeline (#2255). `--draft-block-size` sets the largest proposal and the new `--prompt-lookup-max-batch` (default 2) is the largest active decode batch at which verify rounds run; a paged sequence stops verifying past 4608 total tokens of context (#2264 adjusts the original 4096 cutoff for tokenization and continuation headroom). On GB10 the flag gains 53 to 67 % at B=1 on replies that copy the prompt and costs 3.5 % on chat replies that copy little (#2264 tracks gating those rounds off). Finished requests report the rounds in `timings` with `draft_kind: "prompt-lookup"`. `scripts/bench_serving_concurrency.py` gains `--prompt-style plain|copy`. `mlxcel generate --prompt-lookup` runs the same `Engine::verify_round`.
-- **Experimental AMD GPU (ROCm) backend on Linux** (#1818). `--features rocm` builds against an MLX ROCm backend kept as whole-file overlays in mlxcelverse, copied only when `MLX_BUILD_ROCM` is on, so Metal and CUDA builds compile no ROCm-modified file. Affine, mxfp4 and mxfp8 checkpoints are native; ModelOpt NVFP4 converts to affine 4-bit at load through the new backend quantization capability table (#2030). HIP ports cover FFT through hipFFT, Hadamard, SearchSorted, BitLinear, the Gumbel-max and rejection samplers, the SSM update step, the Mamba1 scan, the fused MoE decode kernels, xIELU, the add3 norm, paged attention, and fused add-RMSNorm and RoPE-append (#2098, #2099, #2100, #2101, #2103, #2107, #2189). A self-hosted gfx1151 runner builds, links and generates in CI (#1989), and `make verify-rocm` runs the gate locally (#2008). See `docs/installation.md#linux-with-amd-rocm-experimental`.
+- **Experimental AMD GPU (ROCm) backend on Linux** (#1818). `--features rocm` builds against an MLX ROCm backend kept as whole-file overlays in mlxcelverse, copied only when `MLX_BUILD_ROCM` is on, so Metal and CUDA builds compile no ROCm-modified file. Affine, mxfp4 and mxfp8 checkpoints are native; ModelOpt NVFP4 converts to affine 4-bit at load through the new backend quantization capability table (#2030). HIP ports cover FFT through hipFFT, Hadamard, SearchSorted, BitLinear, the Gumbel-max and rejection samplers, the SSM update step, the Mamba1 scan, the fused MoE decode kernels, xIELU, the add3 norm, paged attention, and fused add-RMSNorm and RoPE-append (#2098, #2099, #2100, #2101, #2103, #2107, #2189). A self-hosted gfx1151 runner builds, links and generates in CI (#1989), and `make verify-rocm` runs the gate locally (#2008). See `docs/installation.md#linux-with-amd-rocm`.
 - **Bundled WebUI** (`--webui` on `mlxcel serve` and `mlxcel-server`, compiled in by default through the `webui` Cargo feature) (#1851, #1857, #1860, #1865, #1871, #1891, #1894, #2324). It serves a model library with download consent and progress, load, unload and managed-cache delete; bounded streaming local chat; a runtime Activity page; and scoped Settings with next-load profiles. Model-free startup lets the server start with no model loaded. User guide in `docs/webui/README.md` and `docs/webui/user-guide.ko.md`.
 - **`mlxcel generate-image` and `/v1/images/generations`, `/v1/images/edits` behind `--image-model`** (#2497). This is the shared runtime: checkpoint family probe, flow-match schedulers, FLUX.2 VAE, Z-Image DiT blocks, Qwen3 hidden-state conditioner and PNG writer. No model family is registered yet, so every image checkpoint is recognized and then refused (see Known Issues in the release notes and `docs/image-generation.md`).
 - **Nemotron VoiceChat** offline duplex inference (#2037), a cache-aware online session with frame events (#2039), and a `/v1/realtime` WebSocket on both server entry points with base64 PCM16 audio in both directions (#2041). The upgrade enforces the CORS origin policy (#2047), and streaming frame time is under 80 ms (#2054).

CONTRIBUTING.md

@@ -196,7 +196,8 @@ Minimum:
 - Rust **1.97+** (project uses edition 2024)
 - CMake available on `PATH` on both platforms (required by the `mlxcel-core` and `sentencepiece-sys` build scripts)
 - macOS: Apple Silicon Mac on macOS Sonoma+; Xcode Command Line Tools
-- Linux: CUDA 13+ toolchain, OpenBLAS, LAPACK, `pkg-config` (see [`docs/installation.md`](docs/installation.md) for the package list); for the experimental `rocm` feature, a ROCm installation instead of CUDA (see [Linux with AMD ROCm](docs/installation.md#linux-with-amd-rocm-experimental))
+- Linux: CUDA 13+ toolchain, OpenBLAS, LAPACK, `pkg-config` (see [`docs/installation.md`](docs/installation.md) for the package list); for the `rocm` feature, a ROCm installation instead of CUDA (see [Linux with AMD ROCm](docs/installation.md#linux-with-amd-rocm))
+- Windows: Visual Studio 2022+ with the "Desktop development with C++" workload and the `x86_64-pc-windows-msvc` Rust target; for `cuda`, a CUDA 13 toolkit with cuDNN 9 (see [Windows with NVIDIA CUDA](docs/installation.md#windows-with-nvidia-cuda-experimental)); for `rocm`, a ROCm 10.x SDK (see [Windows with AMD ROCm](docs/installation.md#windows-with-amd-rocm))
 
 Recommended local tooling:
 

README.md

@@ -4,19 +4,19 @@
 [![Latest Release](https://img.shields.io/github/v/release/lablup/mlxcel)](https://github.com/lablup/mlxcel/releases/latest)
 [![CI](https://github.com/lablup/mlxcel/actions/workflows/ci.yml/badge.svg)](https://github.com/lablup/mlxcel/actions/workflows/ci.yml)
 
-High-performance LLM, VLM, embedding, reranking, image generation, and audio inference for Apple Silicon and NVIDIA CUDA systems, with experimental AMD ROCm support on Linux. The CLI and server are implemented in Rust and execute MLX SafeTensors checkpoints through native MLX C++ bindings, without Python in the request path or a checkpoint-conversion step.
+High-performance LLM, VLM, embedding, reranking, image generation, and audio inference for Apple Silicon and NVIDIA CUDA systems, with AMD ROCm support on Linux and Windows. The CLI and server are implemented in Rust and execute MLX SafeTensors checkpoints through native MLX C++ bindings, without Python in the request path or a checkpoint-conversion step.
 
 ## Overview
 
 `mlxcel` is both a local inference CLI and a production-oriented model server. It runs text generation, multimodal input, embeddings, reranking, token classification, image generation and editing, speech transcription and synthesis, continuous batching, prompt caching, speculative decoding, and distributed inference in one native runtime.
 
-Apple Silicon is the primary target. Linux/CUDA is supported as a secondary target. AMD GPUs on Linux (ROCm) and native Windows with NVIDIA CUDA are experimental, source-build-only targets, and an opt-in OpenXLA/IREE backend is available as an alpha development path.
+Apple Silicon is the primary target. Linux/CUDA is supported as a secondary target. AMD GPUs on Linux and Windows (ROCm) are supported, source-build-only targets; native Windows with NVIDIA CUDA is experimental; and an opt-in OpenXLA/IREE backend is available as an alpha development path.
 
 ## Current main highlights
 
 The current `main` branch is v0.8.1 plus unreleased work. Install from source to use features that have not reached the latest tagged release; see the [changelog](CHANGELOG.md) for the release-by-release record.
 
-- **Experimental AMD GPU (ROCm) support.** `--features rocm` builds mlxcel on Linux against an MLX ROCm backend vendored into the source tree as the ROCm part of mlxcelverse, applied on the same pinned MLX commit as the Metal and CUDA builds. Affine, mxfp8, and mxfp4 checkpoints, including gpt-oss MoE experts, run on RDNA 3.5 (`gfx1151`); the fused kernels with a HIP port (BitLinear, the samplers, the SSM update step, the Mamba1 scan, the decode-MoE kernels, xIELU, the add3 LayerNorm, and the fused add-RMSNorm and RoPE + append, which are on by default on ROCm) run natively and the rest fall back to MLX graphs. See [Linux with AMD ROCm](docs/installation.md#linux-with-amd-rocm-experimental) for the build and the open gaps.
+- **AMD GPU (ROCm) support, official since 0.8.** `--features rocm` builds mlxcel on Linux and on Windows x86_64 against an MLX ROCm backend vendored into the source tree as the ROCm part of mlxcelverse, applied on the same pinned MLX commit as the Metal and CUDA builds. Affine, mxfp8, and mxfp4 checkpoints, including gpt-oss MoE experts, run on RDNA 3.5 (`gfx1151`); the fused kernels with a HIP port (BitLinear, the samplers, the SSM update step, the Mamba1 scan, the decode-MoE kernels, xIELU, the add3 LayerNorm, and the fused add-RMSNorm and RoPE + append, which are on by default on ROCm) run natively and the rest fall back to MLX graphs. The backend is a source build; see [Linux with AMD ROCm](docs/installation.md#linux-with-amd-rocm) and [Windows with AMD ROCm](docs/installation.md#windows-with-amd-rocm) for the validated configurations and the open gaps.
 - **Bundled WebUI.** `--webui` serves a browser interface from `mlxcel serve` or `mlxcel-server`: a model library with download, load, unload, and delete; local streaming chat; runtime activity; and scoped settings with next-load profiles. See the [WebUI guide](docs/webui/README.md).
 - **Verified `llama-server` compatibility.** A frozen b10621 manifest classifies all 376 pinned options, routes, and native request fields, with no deferred entries. Native completion, embedding, tokenization, template, infill, props, slots, metrics, resumable-stream, router, and LoRA surfaces are implemented or explicitly classified instead of being silently ignored.
 - **OpenAI, Anthropic, and Vertex-compatible serving.** Chat Completions, Completions, Responses, Embeddings, Reranking, Audio, and Anthropic Messages are available alongside the native `llama-server` routes. Optional Vertex AI custom-container routing is supported through the standard `AIP_*` variables.
@@ -267,15 +267,17 @@ Linux/NVIDIA builds require the CUDA toolkit and MLX's CUDA system dependencies:
 cargo build --release --features cuda
 ```
 
-Linux/AMD builds are experimental and need a ROCm installation with its HIP, rocBLAS, rocWMMA, hipBLASLt, and hipRTC CMake packages:
+Linux/AMD builds are supported and need a ROCm installation with its HIP, rocBLAS, rocWMMA, hipBLASLt, and hipRTC CMake packages:
 
 ```bash
 cargo build --release --features rocm
 ```
 
 Native Windows builds with NVIDIA CUDA are experimental and single-node: run `cargo build --release --features cuda` from a Visual Studio developer prompt with `CUDA_PATH` pointing at a CUDA 13 toolkit that has cuDNN 9. See [Windows with NVIDIA CUDA](docs/installation.md#windows-with-nvidia-cuda-experimental).
 
-A plain Linux build has no GPU feature and runs on the CPU, which is not a validated release target and is much slower. See [Installation](docs/installation.md) for the complete prerequisite, CUDA and ROCm architecture, runtime-header, and packaging matrix, and [Linux with AMD ROCm](docs/installation.md#linux-with-amd-rocm-experimental) for the current ROCm status.
+Native Windows builds with AMD ROCm are supported since 0.8 and single-node: run `cargo build --release --features rocm` from the x64 Visual Studio developer prompt with `ROCM_PATH` pointing at a ROCm 10.x SDK. See [Windows with AMD ROCm](docs/installation.md#windows-with-amd-rocm) for the SDK layout and the validated configuration.
+
+A plain Linux build has no GPU feature and runs on the CPU, which is not a validated release target and is much slower. See [Installation](docs/installation.md) for the complete prerequisite, CUDA and ROCm architecture, runtime-header, and packaging matrix, and [Linux with AMD ROCm](docs/installation.md#linux-with-amd-rocm) for the ROCm build and validated configuration.
 
 ## Performance
 
@@ -315,7 +317,9 @@ The split in that table is architectural rather than arbitrary. Across the whole
 
 The VLM figures also moved between mlxcel releases, and unevenly. Re-sweeping that table at one commit left the median row unchanged at 1.00x against its previous reading, with 9 of 71 rows moving more than 10%: `mistral-small-4-119b-2603-4bit` 5.30x, `moondream2` 4.19x, `qwen3-vl-30b-a3b` 2.61x, `qwen3-omni-30b-a3b` 2.57x, then five between 1.16x and 1.46x. The gains come from dtype fixes that reach specific paths, so the aggregate moved from 98% to 105% while most individual rows stayed where they were. Read the per-model tables rather than the median if a particular checkpoint matters.
 
-Per-host sweeps are more current than the combined report. M5 Max and M1 Ultra are both measured on the v0.7.0 commits (2026-09-06 through 2026-09-09) at the same MLX pin and the same pp512/tg128 shape, so cross-host ratios between those two are taken on one version. The per-host records label those runs `0.7.0-beta.1`, which is the crate version string the tree carried until the release commit. Coverage differs by host rather than version: M5 Max carries text, VLM, speculative decoding, batched serving and the embedding and rerank ladder, while M1 Ultra carries text, VLM, speculative decoding and embeddings. GB10 is still on v0.4.0-rc.1 and the older measurement shape, so any ratio involving it mixes versions. Focused reports cover paged attention, MoE, KV compression, embeddings, and speculative decoding. Read [Benchmark results](docs/benchmark_results/model_tests.md), the [benchmark report](docs/benchmark_results/benchmark-report.md), and the [methodology](docs/benchmarks.md) before comparing hosts or planning capacity.
+GB10 has no same-host `mlx-lm` reference, so its record is version-over-version. The 0.8.1 sweep (text on 2026-11-03, VLM on 2026-11-04, pp512/tg128) paired 111 text checkpoints with the 0.4.0-rc.1 record at a **1.026 median decode ratio**, with the 10th percentile at 0.928 and the 90th at 1.364. The 52 VLM pairs read **1.081 median** and 1.012 at the 10th percentile, with no row below the previous record. Prefill is not comparable across those two sweeps because the harness moved to the 512-token synthetic prompt. The same pass fixed and re-measured the three checkpoints that had started failing: the BitNet pair now reads 845.05 and 843.60 tok/s prefill at 140.08 and 191.67 tok/s decode, and DiffusionGemma runs its model-owned diffusion loop at 797.61 prefill and 6.86 decode tok/s. It also added the first embedding and rerank ladder on this host (102 cells, no failures; 9 of 11 text embedders faster than their 2026-08-26 cells) and the image generation ladder (21 text-to-image checkpoints plus 3 edit-only; 4.61 s/image median for generation at 512x512 and 8 steps, 1.83 s fastest).
+
+Per-host sweeps are more current than the combined report. M5 Max and M1 Ultra are both measured on the v0.7.0 commits (2026-09-06 through 2026-09-09) at the same MLX pin and the same pp512/tg128 shape, so cross-host ratios between those two are taken on one version. The per-host records label those runs `0.7.0-beta.1`, which is the crate version string the tree carried until the release commit. GB10 is measured at 0.8.1 on the same pp512/tg128 shape, so a ratio between GB10 and either Apple Silicon host still mixes versions; within GB10, the decode deltas above are version-over-version. Coverage differs by host rather than version: M5 Max carries text, VLM, speculative decoding, batched serving and the embedding and rerank ladder, M1 Ultra carries text, VLM, speculative decoding and embeddings, and GB10 carries text, VLM, embeddings and rerank, and image generation and editing. Focused reports cover paged attention, MoE, KV compression, embeddings, and speculative decoding. Read [Benchmark results](docs/benchmark_results/model_tests.md), the [benchmark report](docs/benchmark_results/benchmark-report.md), and the [methodology](docs/benchmarks.md) before comparing hosts or planning capacity.
 
 MTP performance depends on the target/drafter pair, prompt, verify width, and hardware generation. With the v0.6.0 exactness policy, Gemma 4 12B plus its 4-bit assistant measured 93.2 tok/s against 43.8 tok/s classic decode on M5 Max (**2.13x**, greedy output byte-identical). The faster inexact kernel measured 120.4 tok/s but is not the default. How much that is worth depends heavily on the prompt: on the same host and release, the `speculative_bench --sweep` default 14-token prompt measures the same pairing at 70.9 tok/s against a 45.3 tok/s baseline (**1.57x** at K=4, 35.0% acceptance), against the 2.13x above on a code prompt. The runtime measures exactness and profitability instead of treating either speedup as universal; see [Speculative-decoding acceptance](docs/speculative-acceptance.md), [MTP policy API](docs/mtp-policy-api.md), and the benchmark records under `docs/benchmark_results/`.
 

docs/README.md

@@ -14,7 +14,7 @@ MkDocs pages are added later.
 
 Current GitHub-facing docs:
 
-1. `installation.md` - platform prerequisites and build flags for macOS, Linux with CUDA, Linux with AMD ROCm (experimental), and Windows with NVIDIA CUDA (experimental).
+1. `installation.md` - platform prerequisites and build flags for macOS, Linux with CUDA, Linux with AMD ROCm, Windows with AMD ROCm, and Windows with NVIDIA CUDA (experimental).
 2. `environment-variables.md` - `MLXCEL_*` runtime, build, downloader, cache, and diagnostic knobs.
 3. `benchmarks.md` - benchmark methodology and the requirements for future raw result tables.
 4. `supported-models.md` - maintained architecture/checkpoint support matrix.