Record authoritative Linux comparison baseline

.github/workflows/rust.yml

@@ -50,8 +50,13 @@ jobs:
       - name: Test comparison reporting tools
         run: uv run --project comparisons/tools --locked python -m unittest discover -s comparisons/tools -p 'test_*.py'
       - name: Validate versioned physical-host pair
-        if: hashFiles('comparisons/benchmark-results/macos-aarch64.json', 'comparisons/benchmark-results/linux-x86_64.json') != ''
+        if: hashFiles('comparisons/benchmark-results/macos-aarch64.json') != '' && hashFiles('comparisons/benchmark-results/linux-x86_64.json') != ''
         run: uv run --project comparisons/tools --locked python comparisons/tools/validate_pair.py comparisons/benchmark-results/macos-aarch64.json comparisons/benchmark-results/linux-x86_64.json
+      - name: Regenerate versioned benchmark graphics
+        if: hashFiles('comparisons/benchmark-results/linux-x86_64.json') != ''
+        run: |
+          uv run --project comparisons/tools --locked python comparisons/tools/generate_report.py comparisons/benchmark-results/linux-x86_64.json
+          git diff --exit-code -- comparisons/benchmark-results/LATEST.md comparisons/benchmark-results/graphics
 
   msrv:
     runs-on: ubuntu-latest

TODO.md

@@ -34,13 +34,15 @@ The central hypothesis is:
   hashes, commits, schemas, matrices, or authority status.
 - [x] Track reporting and visualization tooling as a locked uv project with an
   exact Python and Matplotlib environment, isolated from benchmark execution.
-- [ ] Generate deterministic SVG heatmaps, performance profiles, CPU/memory
-  tradeoff views, and an embedded `LATEST.md` from validated report pairs.
+- [x] Generate deterministic SVG heatmaps, a non-lossy performance profile,
+  CPU/memory tradeoff views, and an embedded `LATEST.md` from the authoritative
+  Linux report. Extend the same report rather than opening a separate PR when the
+  clean macOS half becomes available.
 - [x] Add CodSpeed CPU simulation for pull requests and `main`.
 - [ ] Enable public-repository access for the `moderately-ai` Actions runner
   group, set `CODSPEED_MACRO_ENABLED=true`, and validate ARM64 Linux wall time.
-- [ ] Capture and commit the first clean M4 macOS and Ryzen 7950X3D Linux
-  summaries; retain full raw rounds outside Git.
+- [ ] Capture and commit the first clean M4 macOS summary; the Ryzen 7950X3D
+  Linux summary is complete. Retain full raw rounds outside Git.
 - [ ] Revisit the practical-equivalence band only after enough stable repeated
   baselines exist; never tune it in response to a desired result.
 

comparisons/benchmark-results/LATEST.md

@@ -0,0 +1,89 @@
+# Latest benchmark comparison
+
+This is the authoritative `linux-x86_64` baseline. Lower ratios are
+better. CPU classifications and heatmap ratios use `Vec` as the baseline; red
+does not mean an implementation lost to every other candidate. Every measured
+implementation and scenario is retained, and platforms are never pooled.
+
+![CPU performance profile](graphics/linux-x86_64-cpu-profile.svg)
+
+![Complete CPU ratio heatmap](graphics/linux-x86_64-cpu-heatmap.svg)
+
+## What this baseline says
+
+- JackVec is not an across-the-board faster `Vec`: it has
+  6 confidence-qualified wins and
+  7 losses versus `Vec` in this matrix.
+- Its intended nested-density advantage is substantial: requested memory for the
+  empty and sparse nested workloads is
+  0.333× and
+  0.526× Vec,
+  respectively, while each collection owner remains one machine word.
+- The optimized large append path reaches
+  1.013× Vec and
+  0.322× upstream
+  ThinVec. This is a large targeted improvement, not a universal CPU claim.
+- The clearest JackVec CPU losses versus Vec include 1,024-element sequential
+  iteration (1.265×),
+  `u64` retain (1.246×), and
+  four-element append (1.437×).
+  They are retained here as investigation targets.
+- Against the inline candidates, JackVec wins most measured CPU medians, while
+  SmallVec avoids heap allocation when values fit inline. Neither representation
+  dominates every workload.
+
+## CPU outcomes
+
+The confidence-aware classifications below compare each implementation with
+`Vec`. “Inconclusive” means the paired 95% interval crosses a boundary; it is not
+silently counted as equality.
+
+| Implementation | Wins | Equivalent | Inconclusive | Losses |
+|---|---:|---:|---:|---:|
+| JackVec | 6 | 4 | 5 | 7 |
+| ThinVec | 7 | 2 | 3 | 10 |
+| SmallVec4 | 1 | 1 | 3 | 17 |
+| SmallVec8 | 1 | 3 | 3 | 15 |
+
+For direct context, this simpler head-to-head table compares median CPU times
+using the same ±3% practical band. It does not replace the confidence-aware table.
+
+| JackVec compared with | JackVec faster | Within ±3% | JackVec slower |
+|---|---:|---:|---:|
+| Vec | 8 | 5 | 9 |
+| ThinVec | 8 | 7 | 7 |
+| SmallVec4 | 16 | 2 | 4 |
+| SmallVec8 | 15 | 3 | 4 |
+
+## Memory outcomes
+
+Requested and allocator-usable heap are deliberately separate. Requested bytes
+show representation savings; usable bytes show what the measured allocator
+actually retained after size-class rounding.
+
+![Requested live heap ratio heatmap](graphics/linux-x86_64-memory-requested-heatmap.svg)
+
+![Allocator-usable live heap ratio heatmap](graphics/linux-x86_64-memory-usable-heatmap.svg)
+
+Collection-owner size is not included in those heap ratios. A `Vec` owner is 24
+bytes, a JackVec or ThinVec owner is 8 bytes, and SmallVec owners vary with inline
+capacity and element alignment. In nested rows the outer allocation already
+contains every inner owner, so adding the owner column to live heap would double
+count memory. See [the complete platform table](linux-x86_64.md) for
+owner bytes, absolute requested/usable bytes, allocation counts, reallocations,
+and spill counts.
+
+## Run provenance
+
+- Commit: `1f6e5f8abac7152d6ef3523baaed63096ac88b0a`
+- Compiler: `1.97.0` (`2d8144b7880597b6e6d3dfd63a9a9efae3f533d3`)
+- CPU rounds: 5; CPU rows: 110; allocation rows: 60
+- Minimum pinned-core idle audit: 100.0%
+- Maximum audited one-minute load: 1.12
+
+The performance profile reports the fraction of workloads within each factor of
+the fastest implementation for that workload. It is an aggregate view, not a
+claim that all workloads are equally representative. The heatmaps preserve the
+individual results. These microbenchmarks describe the listed operations, element
+types, sizes, compiler, allocator, and machine—not every application. macOS
+remains pending until a clean authoritative run is available.

comparisons/benchmark-results/README.md

@@ -9,3 +9,13 @@ CI or experiment artifacts when investigating a result.
 
 Exploratory runs are automatically suffixed `-exploratory` and must not replace
 an authoritative platform summary.
+
+`LATEST.md` and `graphics/` are deterministic derivatives of the authoritative
+JSON. The report keeps requested heap, allocator-usable heap, and collection-owner
+size conceptually separate; combining them can either hide allocator rounding or
+double-count nested owners. Regenerate it with:
+
+```console
+uv run --project comparisons/tools --locked python comparisons/tools/generate_report.py \
+  comparisons/benchmark-results/linux-x86_64.json
+```