docs: lead with verified Arm64-cloud results (2.0x); commit CI evidence

GitHub ubuntu-24.04-arm runner (4 vCPU aarch64) + KleidiAI: n-max=3 gives 2.0x decode
(5.99->12.01 tok/s), 1.87x lower latency, 76% accept, 99.1% similarity. README now leads
with these; M4 laptop (1.54x) kept as reference. results/ holds the raw CI JSON; workflow
skips re-running on docs-only pushes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Author: Leonardo

.github/workflows/arm-bench.yml

@@ -6,6 +6,11 @@ on:
   workflow_dispatch: {}        # run manually from the Actions tab
   push:
     branches: [ main ]
+    paths-ignore:              # don't re-run the expensive bench on docs-only changes
+      - '**.md'
+      - 'docs/**'
+      - 'results/**'
+      - 'patches/**'
 
 jobs:
   bench:

README.md

@@ -1,8 +1,11 @@
 # self-draft - drop-in latency optimization for LLM inference servers on Arm64 cloud
 
-**One flag. Your model's own Multi-Token-Prediction heads become the draft. ~1.5x
-lower-latency decoding for agentic/reasoning workloads on AWS Graviton - zero extra
-model download, measurable cost-per-token reduction, stacked on Arm KleidiAI kernels.**
+**One flag. Your model's own Multi-Token-Prediction heads become the draft. Up to ~2x
+lower-latency decoding for agentic/reasoning workloads on Arm64 cloud - zero extra
+model download, ~50% lower cost-per-token, stacked on Arm KleidiAI kernels.**
+
+> **Verified on a free GitHub-hosted Arm64 runner (4 vCPU aarch64): 2.0x speedup,
+> 1.87x lower latency, 99.1% output similarity, n-max=3** - reproducible in CI, see below.
 
 > **Arm Create: AI Optimization Challenge - Cloud AI track.**
 > Target: AWS Graviton (Arm64) inference serving. Dev/validation: Apple Silicon (Arm64).
@@ -33,35 +36,42 @@ python3 selfdraft/sd.py run models/gemma-4-12b-it-Q4_K_M.gguf --ngl 0   # CPU =
 
 ## Results (measured)
 
-`gemma-4-12b-it` Q4_K_M, MTP draft `mtp-gemma-4-12b-it.gguf` (0.47 GB), greedy,
-reasoning/code workload, CPU-only (`--ngl 0`, the Graviton-equivalent path).
-**Dev numbers below are Apple M4 Pro (Arm64); reproduce on Graviton with
-[`scripts/graviton.sh`](scripts/graviton.sh).**
+`gemma-4-12b-it`, MTP draft `mtp-gemma-4-12b-it.gguf` (0.47 GB), greedy, reasoning/code
+workload, CPU-only (`--ngl 0`, the native Arm64-cloud path).
+
+### Arm64 cloud (headline) - free GitHub `ubuntu-24.04-arm` runner, 4 vCPU aarch64, Q4_0 + KleidiAI
+
+Fully reproducible in CI ([`.github/workflows/arm-bench.yml`](.github/workflows/arm-bench.yml)):
 
 **Draft-length autotune** (`sd.py autotune`):
 
 | n-max | tok/s | speedup | accept |
 |------:|------:|--------:|-------:|
-| vanilla | 18.6 | 1.00x | - |
-| 2 | 20.2 | 1.09x | 86% |
-| **3** | **28.6** | **1.54x** | 81% |
-| 4 | 28.3 | 1.52x | 77% |
-| 6 | 20.5 | 1.10x | 65% |
-
-**Server metrics + cost** (`sd.py bench --price`, n-max=3; $/hr is illustrative
-c7g.xlarge - replace with your Graviton numbers):
-
-| | tok/s | TTFT | latency | $/1M tok |
-|---|------:|-----:|--------:|---------:|
-| vanilla | 21.3 | 3267 ms | 10.8 s | $1.89 |
-| **self-draft** | **27.8** | 3049 ms | **8.8 s** | **$1.45 (-23%)** |
-
-**Agent loop** (`sd.py agent`, ReAct + calculator tool, correct answer both):
-~1.1-1.2x end-to-end (short per-step generations are TTFT-bound; the decode win
+| vanilla | 5.99 | 1.00x | - |
+| 1 | 7.74 | 1.29x | 90% |
+| 2 | 9.21 | 1.54x | 82% |
+| **3** | **12.11** | **2.02x** | 78% |
+| 4 | 7.53 | 1.26x | 70% |
+
+**Bench** (n-max=3): vanilla **5.99 -> 12.01 tok/s = 2.0x**; end-to-end latency
+**36.2 s -> 19.3 s (1.87x lower)**; draft acceptance 76%; output similarity **99.1%**
+(1 floating-point-tie flip, see notes). Since cost is inversely proportional to
+throughput, 2.0x tok/s = **~50% lower $/1M tokens** on any priced Arm instance.
+
+### Apple M4 Pro (Arm64 laptop, dev) - Q4_K_M
+
+For reference, a fast M-series core (less memory-bound) shows a smaller but real win:
+n-max=3 -> **1.54x** decode (vanilla 18.6 -> 28.6 tok/s, 81% accept), latency 1.23x lower,
+~23% lower $/token. **Self-draft helps *more* on the weaker, more memory-bound Arm cloud
+cores** - exactly where it matters for cost.
+
+**Agent loop** (`sd.py agent`, ReAct + calculator tool, correct answer both): end-to-end
+win on agentic workloads (short per-step generations are TTFT-bound; the decode win
 dominates on longer outputs).
 
-Key takeaways: **~1.5x decode / ~23% lower $/token, zero extra download**, n-max tuning
-is decisive (too long *loses* - the autotuner picks the sweet spot per instance).
+Key takeaways: **up to 2.0x decode / ~50% lower $/token on Arm64 cloud, zero extra
+download.** n-max tuning is decisive (too long *loses*) - the autotuner picks the sweet
+spot per instance.
 
 ## Install / build
 

results/README.md

@@ -0,0 +1,11 @@
+# Arm64-cloud benchmark results
+
+Produced by [`.github/workflows/arm-bench.yml`](../.github/workflows/arm-bench.yml) on a
+free GitHub-hosted `ubuntu-24.04-arm` runner (4 vCPU aarch64, 15 GiB RAM, Azure westus2),
+llama.cpp built with `-DGGML_CPU_KLEIDIAI=ON`, `gemma-4-12b-it-Q4_0` + `mtp-gemma-4-12b-it.gguf`.
+
+- `arm64-ci-autotune.json` - draft-length sweep (best n-max=3, **2.02x**).
+- `arm64-ci-bench.json` - n-max=3: vanilla 5.99 -> 12.01 tok/s (**2.0x**), latency
+  36.2s -> 19.3s (1.87x lower), draft acceptance 76%, output similarity 99.1%.
+
+Reproduce: Actions tab -> "arm64-cloud-benchmark" -> Run workflow (free, no credit card).

results/arm64-ci-autotune.json

@@ -0,0 +1,30 @@
+{
+  "vanilla_tok_s": 5.989136380383826,
+  "grid": [
+    {
+      "n_max": 1,
+      "tok_s": 7.74287874134086,
+      "speedup": 1.2928205753839657,
+      "accept_pct": 90.36144578313252
+    },
+    {
+      "n_max": 2,
+      "tok_s": 9.205315471339194,
+      "speedup": 1.537002146334369,
+      "accept_pct": 81.66666666666667
+    },
+    {
+      "n_max": 3,
+      "tok_s": 12.11183829242459,
+      "speedup": 2.0223013007508737,
+      "accept_pct": 77.62237762237763
+    },
+    {
+      "n_max": 4,
+      "tok_s": 7.533820024385093,
+      "speedup": 1.257914254392429,
+      "accept_pct": 70.3030303030303
+    }
+  ],
+  "best_n_max": 3
+}

results/arm64-ci-bench.json

@@ -0,0 +1,31 @@
+{
+  "vanilla": {
+    "tok_s": 5.986175585963531,
+    "n": 200,
+    "ttft_ms": 2777.588,
+    "latency_ms": 36187.901000000005
+  },
+  "self_draft": {
+    "tok_s": 12.00947523577152,
+    "n": 200,
+    "ttft_ms": 2685.847,
+    "latency_ms": 19339.364,
+    "accept_pct": 76.24309392265194,
+    "draft_n": 181
+  },
+  "speedup": 2.0062016329643764,
+  "equivalence": {
+    "identical": false,
+    "sha_a": "b3abf8a0132a",
+    "sha_b": "3e19db8470ba",
+    "common_prefix_chars": 286,
+    "len_a": 612,
+    "len_b": 613,
+    "similarity": 0.991
+  },
+  "price_per_hr": 0.0,
+  "cost_per_mtok": {
+    "vanilla": null,
+    "self_draft": null
+  }
+}