Repository navigation
Benchmarks: accept ROCm receipts on the board (Strix Halo) #1001
Description
Hello @ashhart,
We would like Strix Halo results on tensorfold.dev/benchmarks. Today the board takes mlx and cuda, and tensorfold benchmark keeps ROCm receipts local. We ran it with the v1.0.12 HIP build (833bbfc) on our Strix Halo (Radeon 8060S, gfx1151), community-v1 at defaults, all cells complete:
| Checkpoint | code T0 | chat T0 | code T1 | chat T1 |
|---|---|---|---|---|
| Qwen3.5-9B-MLX-4bit | 38.83 | 38.95 | 38.61 | 38.51 |
| Qwen3.6-35B-A3B-MLX-4bit-MTP | 103.36 | 79.84 | 80.75 | 72.96 |
| Qwen3.8-27B-MLX-4bit | 12.88 | 13.17 | 12.89 | 12.90 |
| (stream delivery tok/s; first text 0.06-0.50 s; we can attach the receipts.) |
-
The board accepts
rocmonlinux. Once the Worker accepts it, lifting the client's publish refusal is a one-line PR we can send. -
GPU names on ROCm. All three receipts name no GPU. We have a small change ready that reads rocminfo's GPU agents (name, first global pool, compute units; APU = unified). Its tests pass on gfx1151. Shall we open it?
-
The CPU name pattern.
AMD RYZEN AI MAX+ 395 w/ Radeon 8060Shas a/, which is outside the board's pattern, so the CPU is dropped and the receipts arepartial. Allow/, or should the client writewith? -
Local checkpoints. These runs used local copies of the public checkpoints, so they can't be published as they are. We will rerun from the cached repos once the board takes rocm.
-
--drafteron HIP. The HIP server accepts--drafterand doesn't use it: 27B ran at 12.9 tok/s with and without DFlash2, and the receipt doesn't say whether a drafter ran. Should HIP refuse--drafteruntil it supports DFlash2?
Separately: we can run a nightly gfx1151 job on our Strix Halo againstmainonly (never fork PRs), if that helps.
Thank you!
Activity
ashhart commentedon Oct 17, 2026
Thank you, @ThinkOffApp, for three complete gfx1151 runs and the comparison with and without the drafter.
The pinned receipt set from #994 is landing in 1.1.0. The remaining ROCm benchmark/publication work is for 1.1.1, the next patch release; equal timings alone do not show that DFlash2 executed.
Client and board validation need to agree on GPU/CPU identity and checkpoint identity before publication. An unused drafter option also needs a clear refusal until supported. Please keep the full local receipts with those checks.
Posted by Ash's AI Assistant Synthena
ashhart commentedon Oct 18, 2026
Thank you, @ThinkOffApp, for three complete gfx1151 runs and the comparison with and without the drafter.
1.1.0 includes the pinned receipt set through #994 in
3f8fe9c2c. The remaining ROCm benchmark/publication work is for 1.1.1, the next patch release; equal timings alone do not show that DFlash2 executed.Client and board validation need to agree on GPU/CPU identity and checkpoint identity before publication. An unused drafter option also needs a clear refusal until supported. Please keep the full local receipts with those checks.
No new gfx1151 HIP benchmark run is recorded; drafter execution, hardware/checkpoint identity and publication validation remain owed.
Posted by Ash's AI Assistant Synthena
ThinkOffApp commentedon Oct 18, 2026
Thank you @ashhart !
For 1.1.1 we will send two small PRs, each with full local receipts: one makes the benchmark client report GPU, CPU and checkpoint identity the way the board validates them, and one makes the HIP server refuse --drafter clearly until DFlash2 runs there. With them we will add a fresh gfx1151 run whose receipts show drafter execution directly (drafted and accepted token counts per request), not timings alone.
ashhart commentedon Oct 19, 2026
Thanks @ThinkOffApp. The remaining ROCm benchmark and board work is targeted for 1.1.2; the current client still keeps ROCm receipts local.
For #1043-#1045 we still need the physical GPU ordinal fix, wording that distinguishes external DFlash2 from native MTP, checked counter totals, and the fresh gfx1151 identity and per-request drafted/accepted receipts you offered. We will integrate those with matching client and receiver checks before enabling publication; equal timings alone do not show that a drafter ran.
Posted by Ash's AI Assistant Synthena
ThinkOffApp commentedon Oct 19, 2026
Thanks @ashhart !
The four items are now on the branches:
Those GPU runs were made before the visible-device fixes, so they do not exercise them; the #1043 receipts README says so. We will add one fresh gfx1151 run with all three branches merged, attach its receipts, and then mark the three PRs ready for review.
Benchmark draft counts: a fresh gfx1151 run with all three ashhart#1001…
Benchmark draft counts: a fresh gfx1151 run with all three ashhart#1001…
ThinkOffApp commentedon Oct 19, 2026
The fresh gfx1151 run is in #1045, docs/receipts/drafter-counts/fresh-2026-10-19/ (390b17e). One binary with #1043 (c949cc9), #1044 (f2c78ee) and #1045 (134a5f4) merged onto main 76aafcf, on the same Bosgame M5 as before.
One limit: with one GPU, the run with HIP_VISIBLE_DEVICES=0 shows the GPU is kept, but it cannot tell the old probe from the new one; the host controls in #1043 do that.
Benchmark draft counts: a fresh gfx1151 run with all three #1001 bran…