Record authoritative macOS benchmark baseline

.github/workflows/rust.yml

@@ -53,9 +53,9 @@ jobs:
         if: hashFiles('comparisons/benchmark-results/macos-aarch64.json') != '' && hashFiles('comparisons/benchmark-results/linux-x86_64.json') != ''
         run: uv run --project comparisons/tools --locked python comparisons/tools/validate_pair.py comparisons/benchmark-results/macos-aarch64.json comparisons/benchmark-results/linux-x86_64.json
       - name: Regenerate versioned benchmark graphics
-        if: hashFiles('comparisons/benchmark-results/linux-x86_64.json') != ''
+        if: hashFiles('comparisons/benchmark-results/linux-x86_64.json') != '' && hashFiles('comparisons/benchmark-results/macos-aarch64.json') != ''
         run: |
-          uv run --project comparisons/tools --locked python comparisons/tools/generate_report.py comparisons/benchmark-results/linux-x86_64.json
+          uv run --project comparisons/tools --locked python comparisons/tools/generate_report.py comparisons/benchmark-results/linux-x86_64.json comparisons/benchmark-results/macos-aarch64.json
           git diff --exit-code -- comparisons/benchmark-results/LATEST.md comparisons/benchmark-results/graphics
 
   msrv:

TODO.md

@@ -34,14 +34,13 @@ decision is reached.
 - [x] Capture and validate a clean Ryzen 7950X3D Linux matrix from canonical
   `main` merge commit `2dc82e0`, using Rust 1.97.0 and the explicit `system`
   allocator policy, then regenerate the Linux-only `LATEST.md` and SVGs.
-- [ ] Deferred: capture the matching macOS matrix only when dedicated,
-  controllable Apple Silicon hardware is available. The local M4 host is not an
-  acceptable canonical source while interactive iTerm2/WindowServer load prevents
-  the runner from satisfying its idle gate. Never bypass that gate with
-  `--allow-host-noise`.
-- [ ] Deferred with macOS: validate the Linux/macOS pair and extend `LATEST.md` and
-  the SVGs without pooling either platform's absolute or relative values. The
-  absence of a macOS result does not block publishing the canonical Linux result.
+- [x] Capture the matching macOS matrix on the dedicated, normally idle M3 Pro
+  host using the exact Linux source/compiler matrix, AC power, Low Power Mode off,
+  and the strict host-idle gate. Never use `--allow-host-noise` to manufacture a
+  pair.
+- [x] Validate the Linux/macOS pair and extend `LATEST.md` and the SVGs with
+  separate platform sections. Never pool either platform's absolute or relative
+  values.
 - [ ] Enable the already-configured CodSpeed ARM64 macro runner only after public
   repository runner-group access is available; keep it a trend lane, not a
   substitute for physical-host baselines.
@@ -138,9 +137,9 @@ header alignment without a new real-workload counterexample.
 - [x] Track public-repository access for the `moderately-ai` Actions runner in the
   active roadmap; configuration is ready but organization runner-group access and
   `CODSPEED_MACRO_ENABLED=true` remain external prerequisites.
-- [x] Move the matching clean M4 macOS capture and pair validation into the active
-  roadmap; subsequently defer it until dedicated hardware is available, retaining
-  full raw rounds outside Git and publishing no rejected/noisy local result.
+- [x] Capture and validate the matching clean M3 Pro macOS half on dedicated
+  hardware, retain full raw rounds outside Git, and publish only the accepted
+  compact result.
 - [x] Move practical-equivalence-band review into the active roadmap with its
   evidence threshold unchanged.
 
@@ -179,9 +178,9 @@ header alignment without a new real-workload counterexample.
 
 ## Experiment record
 
-### Canonical Linux comparison baseline (`bench/canonical-linux-baseline`)
+### Canonical Linux/macOS comparison baseline (`bench/canonical-linux-baseline`)
 
-- Status: complete on Linux; macOS deliberately deferred
+- Status: complete on Linux and macOS
 - Source: canonical `main` merge commit
   `2dc82e0656ba2c3997c4dc2dd26b905ce621bb2f`; Rust 1.97.0
   (`2d8144b7880597b6e6d3dfd63a9a9efae3f533d3`); explicit system allocator.
@@ -197,14 +196,17 @@ header alignment without a new real-workload counterexample.
   inconclusive results, and six losses versus Vec. Empty and sparse nested
   requested heap are 0.333x and 0.526x Vec. This remains a situational density
   design with targeted CPU wins, not an across-the-board faster Vec.
-- Rejected macOS capture: the local M4 runner correctly refused to proceed when
-  iTerm2 and WindowServer each sustained roughly 28% CPU and two consecutive
-  >=90% idle samples could not be obtained. No local macOS result is authoritative
-  or published. Retry only on dedicated controllable hardware; never use
-  `--allow-host-noise` to manufacture a pair.
-- Decision: publish the canonical Linux result and Linux-only graphics now.
-  Cross-platform validation and graphics remain deferred and do not block Linux
-  performance work; the next credible implementation audit is `retain<u64>`.
+- macOS quality: five balanced rotations on the normally idle M3 Pro,
+  using the exact source commit, Rust 1.97.0 compiler identity, and explicit
+  system allocator. All settle gates passed, no host issue was recorded, and the
+  final noisy post-round sample recovered to 96.7% idle before allocation
+  accounting. The report contains 110 CPU rows and 60 allocation rows.
+- macOS result: JackVec has four confidence-qualified wins, one equivalent,
+  eleven inconclusive results, and six losses versus Vec. `retain_mixed/u64`
+  remains a credible loss at 1.150x and `retain_mixed/64_byte` at 1.070x;
+  reserved resize is a 0.777x win. Results remain platform-specific.
+- Decision: publish the validated pair, platform-specific graphics, and combined
+  non-pooled report. The next credible implementation audit remains `retain<u64>`.
 
 ### Preallocated four-element append (`perf/append-small-audit`)
 

comparisons/README.md

@@ -65,7 +65,9 @@ sustained system load indicates contamination. `validate_pair.py` independently
 refuses reports whose commit, compiler identity, schema, workload matrix, or
 authoritative status differs.
 
-Compact JSON and Markdown summaries in `benchmark-results/` are versioned. Raw
+Compact JSON and Markdown summaries in `benchmark-results/` are versioned. Run
+`generate_report.py` with every validated platform JSON to regenerate a single
+non-pooled `LATEST.md` and the platform-specific graphics. Raw
 per-round estimates and allocation CSV files live under the ignored
 `benchmark-results/raw/` directory.
 
@@ -81,7 +83,7 @@ CodSpeed CPU simulation runs on every pull request and `main` update. Managed
 ARM64 Linux wall time runs on `main` and manual dispatch after the organization
 enables public-repository macro-runner access and sets the repository variable
 `CODSPEED_MACRO_ENABLED=true`. These continuous trends complement rather than
-replace the controlled M4 macOS and x86_64 Linux reports.
+replace the controlled M3 Pro macOS and x86_64 Linux reports.
 
 ## Python tooling
 

comparisons/benchmark-results/LATEST.md

@@ -1,15 +1,20 @@
 # Latest benchmark comparison
 
-This is the authoritative `linux-x86_64` baseline. Lower ratios are
-better. CPU classifications and heatmap ratios use `Vec` as the baseline; red
-does not mean an implementation lost to every other candidate. Every measured
-implementation and scenario is retained, and platforms are never pooled.
+These are the authoritative physical-host baselines. Lower ratios are better.
+CPU classifications and heatmap ratios use `Vec` as the baseline; red does not
+mean an implementation lost to every other candidate. Every measured
+implementation and scenario is retained. Platform results are presented in
+separate sections and are never pooled.
+
+## `linux-x86_64`
+
+This is the authoritative `linux-x86_64` baseline.
 
 ![CPU performance profile](graphics/linux-x86_64-cpu-profile.svg)
 
 ![Complete CPU ratio heatmap](graphics/linux-x86_64-cpu-heatmap.svg)
 
-## What this baseline says
+### What this baseline says
 
 - JackVec is not an across-the-board faster `Vec`: it has
   7 confidence-qualified wins and
@@ -30,7 +35,7 @@ implementation and scenario is retained, and platforms are never pooled.
   SmallVec avoids heap allocation when values fit inline. Neither representation
   dominates every workload.
 
-## CPU outcomes
+### CPU outcomes
 
 The confidence-aware classifications below compare each implementation with
 `Vec`. “Inconclusive” means the paired 95% interval crosses a boundary; it is not
@@ -53,7 +58,7 @@ using the same ±3% practical band. It does not replace the confidence-aware tab
 | SmallVec4 | 13 | 4 | 5 |
 | SmallVec8 | 13 | 4 | 5 |
 
-## Memory outcomes
+### Memory outcomes
 
 Requested and allocator-usable heap are deliberately separate. Requested bytes
 show representation savings; usable bytes show what the measured allocator
@@ -71,7 +76,7 @@ count memory. See [the complete platform table](linux-x86_64.md) for
 owner bytes, absolute requested/usable bytes, allocation counts, reallocations,
 and spill counts.
 
-## Run provenance
+### Run provenance
 
 - Commit: `2dc82e0656ba2c3997c4dc2dd26b905ce621bb2f`
 - Compiler: `1.97.0` (`2d8144b7880597b6e6d3dfd63a9a9efae3f533d3`)
@@ -86,5 +91,91 @@ The performance profile reports the fraction of workloads within each factor of
 the fastest implementation for that workload. It is an aggregate view, not a
 claim that all workloads are equally representative. The heatmaps preserve the
 individual results. These microbenchmarks describe the listed operations, element
-types, sizes, compiler, allocator, and machine—not every application. macOS
-remains pending until a clean authoritative run is available.
+types, sizes, compiler, allocator, and machine—not every application.
+
+## `macos-aarch64`
+
+This is the authoritative `macos-aarch64` baseline.
+
+![CPU performance profile](graphics/macos-aarch64-cpu-profile.svg)
+
+![Complete CPU ratio heatmap](graphics/macos-aarch64-cpu-heatmap.svg)
+
+### What this baseline says
+
+- JackVec is not an across-the-board faster `Vec`: it has
+  4 confidence-qualified wins and
+  6 losses versus `Vec` in this matrix.
+- Its intended nested-density advantage is substantial: requested memory for the
+  empty and sparse nested workloads is
+  0.333× and
+  0.526× Vec,
+  respectively, while each collection owner remains one machine word.
+- The optimized large append path reaches
+  0.854× Vec and
+  0.362× upstream
+  ThinVec. This is a large targeted improvement, not a universal CPU claim.
+- JackVec's three largest median CPU gaps versus Vec are `nested_metadata_scan_sparse` (2.041×), `append_preallocated/4` (1.324×), `retain_mixed/u64` (1.149×). They are
+  retained here as investigation targets; confidence-aware classifications remain
+  authoritative over point-estimate ordering.
+- Against the inline candidates, JackVec wins most measured CPU medians, while
+  SmallVec avoids heap allocation when values fit inline. Neither representation
+  dominates every workload.
+
+### CPU outcomes
+
+The confidence-aware classifications below compare each implementation with
+`Vec`. “Inconclusive” means the paired 95% interval crosses a boundary; it is not
+silently counted as equality.
+
+| Implementation | Wins | Equivalent | Inconclusive | Losses |
+|---|---:|---:|---:|---:|
+| JackVec | 4 | 1 | 11 | 6 |
+| ThinVec | 3 | 3 | 6 | 10 |
+| SmallVec4 | 4 | 2 | 6 | 10 |
+| SmallVec8 | 5 | 2 | 4 | 11 |
+
+For direct context, this simpler head-to-head table compares median CPU times
+using the same ±3% practical band. It does not replace the confidence-aware table.
+
+| JackVec compared with | JackVec faster | Within ±3% | JackVec slower |
+|---|---:|---:|---:|
+| Vec | 6 | 7 | 9 |
+| ThinVec | 7 | 12 | 3 |
+| SmallVec4 | 10 | 6 | 6 |
+| SmallVec8 | 12 | 4 | 6 |
+
+### Memory outcomes
+
+Requested and allocator-usable heap are deliberately separate. Requested bytes
+show representation savings; usable bytes show what the measured allocator
+actually retained after size-class rounding.
+
+![Requested live heap ratio heatmap](graphics/macos-aarch64-memory-requested-heatmap.svg)
+
+![Allocator-usable live heap ratio heatmap](graphics/macos-aarch64-memory-usable-heatmap.svg)
+
+Collection-owner size is not included in those heap ratios. A `Vec` owner is 24
+bytes, a JackVec or ThinVec owner is 8 bytes, and SmallVec owners vary with inline
+capacity and element alignment. In nested rows the outer allocation already
+contains every inner owner, so adding the owner column to live heap would double
+count memory. See [the complete platform table](macos-aarch64.md) for
+owner bytes, absolute requested/usable bytes, allocation counts, reallocations,
+and spill counts.
+
+### Run provenance
+
+- Commit: `2dc82e0656ba2c3997c4dc2dd26b905ce621bb2f`
+- Compiler: `1.97.0` (`2d8144b7880597b6e6d3dfd63a9a9efae3f533d3`)
+- Allocator policy: `system`; inherited injection:
+  `{'DYLD_INSERT_LIBRARIES': None, 'LD_PRELOAD': None}`; effective injection:
+  `{'DYLD_INSERT_LIBRARIES': None, 'LD_PRELOAD': None}`
+- CPU rounds: 5; CPU rows: 110; allocation rows: 60
+- Minimum pinned-core idle audit: 88.2%
+- Maximum audited one-minute load: 2.78
+
+The performance profile reports the fraction of workloads within each factor of
+the fastest implementation for that workload. It is an aggregate view, not a
+claim that all workloads are equally representative. The heatmaps preserve the
+individual results. These microbenchmarks describe the listed operations, element
+types, sizes, compiler, allocator, and machine—not every application.

comparisons/benchmark-results/README.md

@@ -17,5 +17,6 @@ double-count nested owners. Regenerate it with:
 
 ```console
 uv run --project comparisons/tools --locked python comparisons/tools/generate_report.py \
-  comparisons/benchmark-results/linux-x86_64.json
+  comparisons/benchmark-results/linux-x86_64.json \
+  comparisons/benchmark-results/macos-aarch64.json
 ```