From 8131f579bf3bc669cb47f6002fd2d6e76c75ef49 Mon Sep 17 00:00:00 2001
From: Aaron Miller <aaron.miller@algorithmiq.fi>
Date: Mon, 31 Aug 2026 15:16:32 +0100
Subject: [PATCH 01/14] =?UTF-8?q?docs(benches):=20=F0=9F=93=9D=20the=20ben?=
 =?UTF-8?q?chmarking=20ladder=20for=20sensitive=20PRs?=
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Replaces the rung runner with the document it was wrapping. `benches/LADDER.md`
names four groups of benchmark configurations at the sizes the library is used
at — L1 one thread at ~10M terms, L2 one node at ~1B, L3 the same problems on
four nodes, L4 the strong and weak scaling ladders — each as the exact pytest
flags and `-k` selector that produce it, run through the ordinary `just bench`
route. No CLI, no TOML schema, no gate, and nothing to recalibrate but a term
count in a table.

The sizes, term counts and costs are the 2026-08-27 campaign's; time and memory
are rounded to a figure that sizes a job rather than quoted as a baseline.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---
 benches/LADDER.md                | 240 +++++++++++++++++++++++++++++++
 benches/README.md                |   4 +-
 benches/conftest.py              |   6 +-
 docs/content/docs/benchmarks.mdx | 160 +++++++++++++++++++++
 4 files changed, 407 insertions(+), 3 deletions(-)
 create mode 100644 benches/LADDER.md

diff --git a/benches/LADDER.md b/benches/LADDER.md
new file mode 100644
index 00000000..7aabdb5c
--- /dev/null
+++ b/benches/LADDER.md
@@ -0,0 +1,240 @@
+# The benchmarking ladder for sensitive PRs
+
+Four groups of benchmark configurations at the sizes monoprop is used at. Each row gives the
+pytest flags and the `-k` selector that produce it, so a group is a handful of ordinary
+`just bench` invocations — there is no separate runner, no table format and no gate.
+
+**Nothing runs these for you, and no benchmark gates a pull request.** Run the group your change
+could plausibly move, on your branch **and** on its merge base, and put both sets of numbers in the
+PR. Quote the `_core.so` md5 from each run's `meta` block: it is the only thing that identifies
+which binary produced a number.
+
+## Which group
+
+| a change to | run |
+| --- | --- |
+| a kernel, encoding or data structure | L1, then L2. Hubbard makes 29 `propagate` calls and pauli 1, so a per-call cost shows on one and hides on the other |
+| the graph build or contraction | L2's graph and eval rows, both pictures |
+| MPI, routing, exchange or placement | L4. A communication cost is a function of `P` = ranks × partitions, not of node count |
+| memory layout or allocation | L1 **and** L2. The per-term and the per-process terms of the memory cost only separate across sizes |
+| ranks per node, or anything paid per process | L2 against L3 — same cores per node, 1 rank against 8 |
+
+## Running one row
+
+```bash
+uv sync --all-groups --all-extras          # once
+just bench L1-hubbard --hubbard-cutoff=10 --hubbard-lower-atol=4.2e-05 \
+    -k "test_model_propagate and hubbard"
+```
+
+`LABEL` is one column in `benches/results/REPORT.md`, so name it for the group, the row and the
+arm: `L1-hubbard-branch`, `L1-hubbard-base`. Each run writes `results/time-<label>.json`
+(timings) and `results/<label>.json` (term counts, peak RSS, operator memory), then rebuilds the
+report. Reps are repeated invocations under different labels; leave `--bench-rounds` at 1, since
+`pedantic` builds round *k+1*'s arguments before releasing round *k*'s and a second round holds two
+propagators.
+
+Run one group of operations per process at scale: `build_graph`/`propagate` and `energy`/`gradient`
+each hold their own operator, and selecting all four holds two per rank.
+
+**Terms are exact; time and memory are not.** A term count is a property of the flags alone and is
+identical at every rank and partition count (`cpp/tests/partition_equivalence_tests.cpp`) — so it
+transfers to your machine and a disagreement means the *problem* changed. The `~s` and `~GiB/node`
+columns are one measurement on 128-core, 242 GiB nodes, rounded to a figure you can size a job
+with. They are not a baseline; your baseline is the merge base measured next to your branch.
+
+## L1 — one thread, ~10M terms
+
+One node, one rank, one partition: `export monoprop_PARTITIONS=1 monoprop_NUM_THREADS=1`.
+Removes every threading and MPI effect, so what moves is the kernel.
+
+| row | flags | `-k` | terms | ~s | ~GiB |
+| --- | --- | --- | ---: | ---: | ---: |
+| hubbard `propagate` | `--hubbard-cutoff=10 --hubbard-lower-atol=4.2e-05` | `test_model_propagate and hubbard` | 9,953,109 | 30 | 1 |
+| pauli `propagate` | `--pauli-cutoff=12 --pauli-lower-atol=1.22e-04` | `test_model_propagate and pauli` | 10,069,308 | 20 | 1 |
+
+## L2 — one node, ~1B terms
+
+One node, one rank, every core: `export monoprop_PARTITIONS=128 monoprop_NUM_THREADS=128`
+(substitute your core count). The random rows are the only ones carrying all four operations, so
+they are where the operations separate.
+
+| row | flags | `-k` | terms | ~s | ~GiB |
+| --- | --- | --- | ---: | ---: | ---: |
+| hubbard `propagate` | `--hubbard-cutoff=10 --hubbard-lower-atol=3.38e-06` | `test_model_propagate and hubbard` | 1,001,661,534 | 140 | 60 |
+| pauli `propagate` | `--pauli-cutoff=14 --pauli-lower-atol=8.9e-06` | `test_model_propagate and pauli` | 985,970,588 | 90 | 80 |
+| random `propagate` | *(random flags below)* | `test_random_propagate and heisenberg` | 948,937,993 | 50 | 95 |
+| random `build_graph` | *(same)* | `test_random_build_graph and heisenberg` | 948,937,993 | 50 | 110 |
+| random `energy`+`gradient` | *(same)* | `(test_random_energy or test_random_gradient) and heisenberg` | 948,937,993 | 40 | 110 |
+| Schrödinger `build_graph` | *(same)* | `test_random_build_graph and schrodinger` | 18,825,440 | 1.5 | 17 |
+| Schrödinger `energy`+`gradient` | *(same)* | `(test_random_energy or test_random_gradient) and schrodinger` | 18,825,440 | 0.3 | 11 |
+
+The random flags, identical on every random row:
+
+```
+--num-generators=1000 --num-modes=142 --cutoff=6 --obs-terms=14750000
+```
+
+`--obs-terms` is the size knob in the Heisenberg picture, and at 1000 generators it multiplies by
+about 64 on the way to the propagated operator (14.75M observable terms → 948,937,993) — so walk
+**up** from a small observable, because an overshoot is an OOM rather than a slow run. It is
+**not** a size knob in the Schrödinger picture, which evolves the state: 200k, 2M and 14.75M
+observable terms all give ~19M. Nor can the other axes compensate — `--cutoff 8` and
+`--num-modes 200` each exceeded a 242 GiB node. ~19M is the picture's ceiling, measured.
+
+At ~949M terms on one rank over 128 cores the four operations cost 0.100, 0.107, 0.115 and 0.115
+GiB per Mterm (`propagate`, `energy`, `build_graph`, `gradient`) — within 15% of each other, so no
+operation is a multiple of another.
+
+## L3 — four nodes, the same problems
+
+Same flags, same term counts, same cores per node as L2; 8 ranks per node × 16 partitions instead
+of 1 × 128, so `R` = 32 and `P` = 512. Directly comparable to L2 row for row, which is the point of
+having both.
+
+| row | ~s | ~GiB/node | against L2 |
+| --- | ---: | ---: | --- |
+| hubbard `propagate` | 35 | 18 | 4.1x faster, 3.2x less memory |
+| pauli `propagate` | 23 | 25 | 4.1x faster, 3.3x less memory |
+| random `propagate` | 13 | 100 | 3.8x faster, **0.95x — slightly worse** |
+| random `build_graph` | 13 | 100 | 3.9x faster, **1.05x — flat** |
+| random `energy`+`gradient` | 9 | 60 | 4.3x faster, 1.8x less memory |
+| Schrödinger `build_graph` | 1.1 | 80 | 1.3x faster, **4.8x more memory** |
+| Schrödinger `energy`+`gradient` | 0.15 | 40 | 2.0x faster, **3.5x more memory** |
+
+Time partitions across nodes; memory does not always. The cause is a **~9.5 GiB per-process
+resident floor**, paid once per rank, so eight ranks on a node pay it eight times: on random
+`build_graph` that is 75.6 of the measured 104.2 GiB/node before the build allocates anything.
+More ranks per node buys speed and costs a floor, and it hurts most where the operation's own cost
+is smallest — which is why the 19M-term Schrödinger rows cost more per node on four nodes than on
+one. The benchmarks page has the per-rank breakdown.
+
+## L4 — strong and weak scaling
+
+Hubbard `propagate` only: a communication cost is a function of `P`, so varying the model as well
+would confound it. 8 ranks per node × 16 partitions throughout, so `R` = 8N and `P` = 128N.
+
+Size is set by `--hubbard-lower-atol` at a fixed `--hubbard-cutoff=10`. The cutoff is saturated
+there — at `atol=1.25e-05`, cutoffs 10/12/14 give 96.98M/105.79M/106.82M terms — so all growth
+comes from the tolerance and every point stays inside the same operator family. The sequence
+doubles, which is what lets a cell be shared by both ladders:
+
+| k | `--hubbard-lower-atol` | terms |
+| ---: | --- | ---: |
+| 0 | `1.25e-05` | 96,981,051 |
+| 1 | `8.8e-06` | 184,124,520 |
+| 2 | `5.9e-06` | 377,482,074 |
+| 3 | `3.9e-06` | 781,669,404 |
+| 4 | `2.6e-06` | 1,569,152,761 |
+| 5 | `1.73e-06` | 3,104,527,573 |
+| 6 | `1.14e-06` | 6,125,805,627 |
+| 7 | `7.35e-07` | 12,255,330,837 |
+| 8 | `4.68e-07` | 24,419,998,198 |
+| 9 | `2.94e-07` | 48,317,129,677 |
+| 10 | `1.82e-07` | 94,684,031,363 |
+
+Above k=6 every size was calibrated by measurement: the local exponent
+`d(log terms)/d(-log atol)` drifts from 1.83 at k=1 to 1.40 at k=10, so a power law fitted at one
+end does not predict the other.
+
+**Weak** — hold terms per node constant: **step one row down the table each time you double the
+node count.** Three ladders, each seven points from N=1 to N=64:
+
+| ladder | starts at | Mterms/node | ~s at N=1 → N=64 |
+| --- | --- | ---: | --- |
+| ~97M/node | k=0 at N=1 | 97 | 11 → 37 |
+| ~385M/node | k=2 at N=1 | 385 | 48 → 80 |
+| ~1529M/node | k=4 at N=1 | 1529 | 216 → 257 |
+
+That spread **is** the weak-scaling result; ideal scaling would be flat. Efficiency is
+`t(N=1)/t(N)`.
+
+**Strong** — hold one k and vary N. Three ladders:
+
+| ladder | k | terms | N | ~s at smallest N → N=64 |
+| --- | ---: | ---: | --- | --- |
+| 1.57e9 fixed | 4 | 1,569,152,761 | 1…64 | 216 → 24 |
+| 6.13e9 fixed | 6 | 6,125,805,627 | 2…64 | 450 → 37 |
+| 2.44e10 fixed | 8 | 24,419,998,198 | 8…64 | 471 → 80 |
+
+Efficiency against the ladder's smallest node count `N0` is `N0 · t(N0) / (N · t(N))`. The larger
+two ladders start above one node because the problem does not fit below it, not because the point
+was skipped. Size a point with
+
+```
+GiB/node ≈ 3.4 + 0.063 × Mterms/node
+```
+
+fitted over the campaign's 38 points and specific to hubbard `propagate` at 8 ranks × 16
+partitions — a 242 GiB node holds about 3000 Mterms/node. The law does not transfer to another
+model: the random model measures 0.100–0.115 GiB/Mterm against this slope's 0.063.
+
+The measured answer already exists on the benchmarks page, so a new campaign is a diff against it
+rather than a first measurement. Its headline: each doubling of the problem buys one more useful
+doubling of nodes, and a run that can afford ~1.5e9 terms per node weak-scales at 84% to 8192
+cores while one at ~100M/node reaches 29%.
+
+## Shapes
+
+Three numbers: nodes `N`, ranks per node `R`, partitions per rank `P`, one core per partition —
+so `R × P` is the cores per node. `P` is an environment variable, not a pytest flag.
+
+One rank over a whole node — L1 (with `P`=1) and L2:
+
+```bash
+#!/bin/bash
+#SBATCH --nodes=1 --ntasks-per-node=1 --cpus-per-task=128 --exclusive --mem=0 --time=0:30:00
+set -euo pipefail
+export monoprop_PARTITIONS=128 monoprop_NUM_THREADS=128
+just bench L2-hubbard-branch --hubbard-cutoff=10 --hubbard-lower-atol=3.38e-06 \
+    -k "test_model_propagate and hubbard"
+```
+
+`N` nodes at 8 ranks × 16 partitions — L3 and L4. This is the same command with `srun` inserted
+and the report moved out of it:
+
+```bash
+#!/bin/bash
+#SBATCH --nodes=4 --ntasks-per-node=8 --cpus-per-task=16 --exclusive --mem=0 --time=1:00:00
+set -euo pipefail
+export monoprop_PARTITIONS=16 monoprop_NUM_THREADS=16
+export monoprop_BENCH_LABEL=L3-hubbard-branch monoprop_BENCH_RESULTS=benches/results
+srun --ntasks-per-node=8 --cpus-per-task=16 --cpu-bind=cores \
+    uv run --no-sync python -m pytest benches -o filterwarnings=default \
+    --benchmark-json="benches/results/time-$monoprop_BENCH_LABEL.json" \
+    -k "test_model_propagate and hubbard" \
+    --hubbard-cutoff=10 --hubbard-lower-atol=3.38e-06
+uv run --no-sync monoprop-bench-report benches/results
+```
+
+The report runs **outside** `srun`: under it, one process per rank would race to write the same
+file. The two JSON artifacts are safe because only rank 0 writes them (`conftest.py`
+`pytest_configure`). A ladder is a loop over sizes inside one allocation sized for its largest
+point.
+
+## Traps
+
+Each of these produces a plausible wrong number rather than an error.
+
+- **`srun --cpu-bind=cores` with no `--cpus-per-task` on the `srun` confines each task to one
+  core.** Measured in one allocation: no flags → 128 CPUs, `--cpus-per-task=128` → 128,
+  `--cpu-bind=cores` alone → 1. Repeat both flags on the `srun`, not only on the `#SBATCH`.
+- **Export `monoprop_PARTITIONS`.** The engine's own default is `ranks == 1 ? cores : 1`, so an
+  unset knob on a multi-rank run measures one partition per rank at a wall time plausible enough to
+  publish.
+- **`--cpu-bind=none` cost 1.45x** against Slurm's own cgroup pinning at 8 ranks per node. With one
+  rank holding a whole node there is nothing to confine and it stops mattering.
+- **Leave `MALLOC_ARENA_MAX` unset.** Setting it to the partition count cost ~16% of wall.
+- **`nproc` lies inside a job** — it honours `OMP_NUM_THREADS`, which the BLAS pinning sets to 1,
+  so a job holding 128 CPUs can log one core. Read `os.sched_getaffinity(0)`.
+- **Peak RSS follows capacity, not live bytes**, so a `shrink_to_fit` can raise it: `realloc` holds
+  the old and the new buffer at once.
+- **`--hubbard-observable-site` (46) must stay below `--hubbard-num-sites` (60).** Shrinking the
+  model without moving the observable fails.
+
+## When a term count moves
+
+Nothing here refuses a run, so read the term count yourself. A count that differs from this file is
+an **accuracy change**, and it is the only signal that a timing win was not one. Say so in the PR,
+and update the row in the same PR. Recalibrating a row costs one cheap run: the term count is the
+same at every shape, so measure it at the smallest one.
diff --git a/benches/README.md b/benches/README.md
index 32eb8dec..852d1ac2 100644
--- a/benches/README.md
+++ b/benches/README.md
@@ -11,7 +11,9 @@ documentation for detailed instructions.
 ## What lives where
 
 This directory holds only monoprop's own benchmarks — `conftest.py` (the fixtures
-and the results schema), `bench_random.py`, `bench_models.py`, and `results/`.
+and the results schema), `bench_random.py`, `bench_models.py`,
+[`LADDER.md`](LADDER.md) (the benchmark ladder for sensitive pull requests: the
+groups, their flags and their shapes) and `results/`.
 
 Both bench modules measure the same four operations — `build_graph`, `propagate`,
 `energy` and `gradient` — so a number means the same thing whichever problem
diff --git a/benches/conftest.py b/benches/conftest.py
index b2af8acd..8c44bb1d 100644
--- a/benches/conftest.py
+++ b/benches/conftest.py
@@ -136,8 +136,10 @@ def _spread(comm: Any, value: int) -> dict[str, int]:
 _RESULTS: dict[str, Any] = {
     "meta": {},  # run configuration (ranks, threads, host, ...)
     "params": {},  # resolved random-problem hyperparameters
-    "memhwm": {},  # node id -> summed peak RSS, whole test, setup() included
-    "memhwm_max": {},  # node id -> worst-rank peak RSS, whole test, setup() included
+    # Keyed by PYTEST node id, so a two-operation row gets one entry each and the windows
+    # overlap: max over them, never sum (each already contains the resident operator).
+    "memhwm": {},  # pytest node id -> summed-over-ranks peak RSS, whole test, setup() included
+    "memhwm_max": {},  # pytest node id -> worst-rank peak RSS, whole test, setup() included
     "opsize": {},  # picture / model / node id -> {"terms": n}
     "configs": {},  # fixed model -> config dataclass fields
     "opmem": {},  # fixed model -> per-field operator memory split (bytes)
diff --git a/docs/content/docs/benchmarks.mdx b/docs/content/docs/benchmarks.mdx
index 36326af1..7e910874 100644
--- a/docs/content/docs/benchmarks.mdx
+++ b/docs/content/docs/benchmarks.mdx
@@ -299,6 +299,161 @@ just bench-build-mpi                           # build once (MPI on)
 monoprop_NUM_THREADS=2 just bench-mpi r5t2 5 --map-by slot:PE=2 --bind-to core
 ```
 
+### The ladder
+
+The default sizes above fit a laptop and a CI runner; the sizes the library is used at do not.
+`benches/LADDER.md` is the set at those sizes, run through the ordinary
+`just bench LABEL <flags>` route (or that same pytest line under `srun` for multi-rank): four
+groups, each a table of the exact flags and `-k` selector that produce it. L1 is one thread at
+~10M terms; L2 is one node at ~1B; L3 is the same problems on four nodes at 8 ranks/node x 16
+partitions; L4 is the strong and weak scaling ladders of hubbard `propagate`. The document also
+gives the sbatch shapes for each group and the pinning traps that silently unpin a run.
+
+Nothing runs these for you, and no benchmark gates a pull request. Running the group a change
+could plausibly move, and putting the numbers in the pull request, is the author's job.
+
+### Scaling
+
+Node scaling of Hubbard `propagate`, 1 to 64 nodes (128 to
+8192 cores) at 8 MPI ranks per node × 16 partitions, so `R = 8N` and `P = 128N` with one core per
+partition. Measured 2026-08-27 on AMD EPYC 7742 nodes with 242.0 GiB usable each; 38 points, five
+reps apiece (ten on the smallest weak ladder), every rep gate-checked on binary identity,
+environment and term count before its time was kept. `benches/LADDER.md`'s L4 group reproduces
+these points; the medians below are what a run of it is a diff against.
+
+The measuring binary was `perf/linear-routing-on-wire` @ `c2554554` (`_core.so` md5
+`1157a5e2b421fb8bd18fc6d16fa39778`), whose default is GF(2)-linear on-wire routing. That is the
+branch's default, **not** `main`'s: the figures are internally consistent and traceable, but they
+are not a measurement of `main`.
+
+Problem size is `--hubbard-lower-atol` at a fixed cutoff of 10, which is saturated there, so all
+growth comes from the tolerance and every point stays inside the same operator family. The eleven
+sizes form a doubling sequence and both families are built from it, so a cell reached by two
+ladders is *exactly* the same problem — nine such cells agreed to within 1.24%, most within 0.2%,
+which is a free check on the whole apparatus.
+
+Some points are absent because the problem does not fit in that many nodes' memory, not because
+they were skipped: `strong-6126m` starts at two nodes and `strong-24420m` at eight.
+
+**`strong-1569m`** — 1,569,152,761 terms held fixed.
+
+| nodes | R | P | terms | Mterms/node | median s | Mterms/s/node | GiB/node |
+|---:|---:|---:|---:|---:|---:|---:|---:|
+| 1 | 8 | 128 | 1,569,152,761 | 1569 | 215.61 | 7.28 | 99.9 |
+| 2 | 16 | 256 | 1,569,152,761 | 785 | 105.91 | 7.41 | 51.5 |
+| 4 | 32 | 512 | 1,569,152,761 | 392 | 53.34 | 7.35 | 28.0 |
+| 8 | 64 | 1024 | 1,569,152,761 | 196 | 28.58 | 6.86 | 15.9 |
+| 16 | 128 | 2048 | 1,569,152,761 | 98 | 18.12 | 5.41 | 11.0 |
+| 32 | 256 | 4096 | 1,569,152,761 | 49 | 16.13 | 3.04 | 6.6 |
+| 64 | 512 | 8192 | 1,569,152,761 | 25 | 23.99 | 1.02 | 4.1 |
+
+**`strong-6126m`** — 6,125,805,627 terms held fixed.
+
+| nodes | R | P | terms | Mterms/node | median s | Mterms/s/node | GiB/node |
+|---:|---:|---:|---:|---:|---:|---:|---:|
+| 2 | 16 | 256 | 6,125,805,627 | 3063 | 450.19 | 6.80 | 200.2 |
+| 4 | 32 | 512 | 6,125,805,627 | 1531 | 224.45 | 6.82 | 101.3 |
+| 8 | 64 | 1024 | 6,125,805,627 | 766 | 109.53 | 6.99 | 51.3 |
+| 16 | 128 | 2048 | 6,125,805,627 | 383 | 58.75 | 6.52 | 27.4 |
+| 32 | 256 | 4096 | 6,125,805,627 | 191 | 38.09 | 5.03 | 16.1 |
+| 64 | 512 | 8192 | 6,125,805,627 | 96 | 36.95 | 2.59 | 11.4 |
+
+**`strong-24420m`** — 24,419,998,198 terms held fixed.
+
+| nodes | R | P | terms | Mterms/node | median s | Mterms/s/node | GiB/node |
+|---:|---:|---:|---:|---:|---:|---:|---:|
+| 8 | 64 | 1024 | 24,419,998,198 | 3052 | 471.03 | 6.48 | 201.3 |
+| 16 | 128 | 2048 | 24,419,998,198 | 1526 | 235.19 | 6.49 | 101.8 |
+| 32 | 256 | 4096 | 24,419,998,198 | 763 | 127.10 | 6.00 | 52.2 |
+| 64 | 512 | 8192 | 24,419,998,198 | 382 | 79.58 | 4.79 | 26.9 |
+
+**`weak-97m`** — ~97M terms per node.
+
+| nodes | R | P | terms | Mterms/node | median s | Mterms/s/node | GiB/node |
+|---:|---:|---:|---:|---:|---:|---:|---:|
+| 1 | 8 | 128 | 96,981,051 | 97 | 10.75 | 9.02 | 10.8 |
+| 2 | 16 | 256 | 184,124,520 | 92 | 10.58 | 8.70 | 9.6 |
+| 4 | 32 | 512 | 377,482,074 | 94 | 13.13 | 7.19 | 11.5 |
+| 8 | 64 | 1024 | 781,669,404 | 98 | 14.54 | 6.72 | 10.9 |
+| 16 | 128 | 2048 | 1,569,152,761 | 98 | 18.11 | 5.42 | 11.0 |
+| 32 | 256 | 4096 | 3,104,527,573 | 97 | 24.39 | 3.98 | 11.0 |
+| 64 | 512 | 8192 | 6,125,805,627 | 96 | 36.59 | 2.62 | 11.4 |
+
+**`weak-385m`** — ~385M terms per node.
+
+| nodes | R | P | terms | Mterms/node | median s | Mterms/s/node | GiB/node |
+|---:|---:|---:|---:|---:|---:|---:|---:|
+| 1 | 8 | 128 | 377,482,074 | 377 | 47.71 | 7.91 | 26.6 |
+| 2 | 16 | 256 | 781,669,404 | 391 | 50.90 | 7.68 | 28.0 |
+| 4 | 32 | 512 | 1,569,152,761 | 392 | 53.29 | 7.36 | 27.9 |
+| 8 | 64 | 1024 | 3,104,527,573 | 388 | 54.73 | 7.09 | 27.5 |
+| 16 | 128 | 2048 | 6,125,805,627 | 383 | 58.03 | 6.60 | 27.4 |
+| 32 | 256 | 4096 | 12,255,330,837 | 383 | 64.64 | 5.92 | 26.3 |
+| 64 | 512 | 8192 | 24,419,998,198 | 382 | 79.89 | 4.78 | 26.9 |
+
+**`weak-1529m`** — ~1529M terms per node.
+
+| nodes | R | P | terms | Mterms/node | median s | Mterms/s/node | GiB/node |
+|---:|---:|---:|---:|---:|---:|---:|---:|
+| 1 | 8 | 128 | 1,569,152,761 | 1569 | 215.61 | 7.28 | 99.9 |
+| 2 | 16 | 256 | 3,104,527,573 | 1552 | 222.27 | 6.98 | 100.2 |
+| 4 | 32 | 512 | 6,125,805,627 | 1531 | 224.04 | 6.84 | 101.2 |
+| 8 | 64 | 1024 | 12,255,330,837 | 1532 | 227.33 | 6.74 | 99.9 |
+| 16 | 128 | 2048 | 24,419,998,198 | 1526 | 235.55 | 6.48 | 101.8 |
+| 32 | 256 | 4096 | 48,317,129,677 | 1510 | 249.62 | 6.05 | 101.3 |
+| 64 | 512 | 8192 | 94,684,031,363 | 1479 | 257.05 | 5.76 | 84.3 |
+
+
+Two things follow from the strong curves, and only from having three of them. The reversal in
+`strong-1569m` at 64 nodes is not a property of a core count: the same hardware still improves at
+64 nodes on the two larger problems. What sets the optimum is the load *per node*, and it falls
+below roughly 50–100 Mterms/node at the point each curve turns. Each doubling of the problem buys
+one more useful doubling of nodes.
+
+Weak scaling says the same thing from the other side. At ~97M terms/node, 64 nodes runs at 29% of
+one node's efficiency; at ~385M it is 60%; at ~1529M it is 84% across 64× the hardware. A run that
+can afford about 1.5e9 terms per node weak-scales almost ideally to 8192 cores; one at 100M/node
+does not.
+
+Peak resident memory over all 38 points, spanning 25 to 3063 Mterms per node:
+
+```
+GiB/node = 3.39 + 0.0634 × Mterms/node
+```
+
+The law is **specific to hubbard `propagate` at 8 ranks x 16 partitions** and does not transfer:
+the random model at ~949M terms measures 0.100 GiB/Mterm propagating, 0.115 building the graph
+and 0.115 evaluating energy and gradient, against this slope's 0.063. Both terms of the fit are
+real. The slope is the per-term cost, about 68 bytes per term. The constant is a
+genuine per-node floor that a model through the origin cannot reproduce — the smallest point
+measures 4.1 GiB/node at 25 Mterms/node. Every point falls within −2.9 to +4.5 GiB/node of that
+line except `weak-1529m` at 64 nodes, which measures 84.3 against 97.1 predicted, identically on
+all five reps; it is reproducible and unexplained, and no mechanism is claimed for it on the
+strength of one point. The largest cell measured 201.3 GiB/node against a 242.0 GiB grant.
+
+Memory partitions across nodes for hubbard and pauli and **does not** for the random model, whose
+per-node peak barely moves on any operation. The cause is a per-process floor. For the random
+`build_graph` row of L2 against L3, `opmembase` is the resident floor when the timed call starts
+and `opmemdelta` is the call's own cost:
+
+| per rank | 1 node, R=1 | 4 nodes, R=32 |
+| --- | ---: | ---: |
+| floor (`opmembase`) | 13.04 GiB | 9.45 GiB |
+| the call itself (`opmemdelta`) | 96.25 GiB | 3.59 GiB |
+
+Both columns reconcile to the measured peak exactly: `13.0 + 96.3 = 109.3` GiB/node, and
+`8 x 9.45 + 8 x 3.59 = 104.2`. The operation's own memory partitions almost perfectly -- 27x less
+per rank for 32x the ranks -- while the floor is a **~9.5 GiB per-process constant**, so eight
+ranks on a node pay it eight times and 75.6 of that row's 104.2 GiB/node is floor before the graph
+build allocates anything. It is not per-partition (128 partitions carry 13.04 GiB, 16 carry 9.45)
+and it is not the operator: every field of `opmembreak` partitions cleanly, its rank-summed total
+moving only 88.0 to 86.0 GiB from 1 rank to 32. What the 9.5 GiB consists of is **not attributed**
+by any recorded field; allocator-retained construction memory, fabric buffers and per-process
+thread state are the unmeasured candidates. The consequence is the useful part: **more ranks per
+node buys speed and costs a floor**, paid per process, so it hurts most where the operation's own
+cost is smallest -- worst on the 19M-term Schrodinger rows, where four nodes cost more per node
+than one.
+
 ### Continuous benchmarking
 
 The `bench_main.yml` workflow benchmarks every commit on `main`. [Bencher](https://bencher.dev/)
@@ -339,6 +494,11 @@ The bare-metal workflow provisions a dedicated node for each run. Its lower meas
 supports tighter timing thresholds and permits execution of the fixed models omitted on the shared
 runner.
 
+**No benchmark workflow gates a pull request.** Both track the main branch, and a threshold
+over a series that spans runners is a report, not a verdict. Measuring a change is the author's
+job -- see "The ladder" above. Every run uploads `bmf.json` and the raw result JSON as an artifact,
+so a failed Bencher upload does not take the measurement with it.
+
 The `bench-markers` and `bench-rounds` inputs of the reusable `bench.yml` workflow select the
 benchmarks and repetition count. These values should remain fixed after establishing a testbed,
 because changing them changes the meaning of its time series:

From f86c303869e9e2213c109e353bd19735966267eb Mon Sep 17 00:00:00 2001
From: Aaron Miller <61472721+diagonal-hamiltonian@users.noreply.github.com>
Date: Mon, 31 Aug 2026 22:13:02 +0100
Subject: [PATCH 02/14] Revise LADDER.md for improved benchmarking clarity

Updated the benchmarking ladder documentation to enhance clarity and accuracy regarding benchmark configurations and their usage.

Signed-off-by: Aaron Miller <61472721+diagonal-hamiltonian@users.noreply.github.com>
---
 benches/LADDER.md | 77 ++++-------------------------------------------
 1 file changed, 6 insertions(+), 71 deletions(-)

diff --git a/benches/LADDER.md b/benches/LADDER.md
index 7aabdb5c..100ff5cd 100644
--- a/benches/LADDER.md
+++ b/benches/LADDER.md
@@ -1,13 +1,4 @@
-# The benchmarking ladder for sensitive PRs
-
-Four groups of benchmark configurations at the sizes monoprop is used at. Each row gives the
-pytest flags and the `-k` selector that produce it, so a group is a handful of ordinary
-`just bench` invocations — there is no separate runner, no table format and no gate.
-
-**Nothing runs these for you, and no benchmark gates a pull request.** Run the group your change
-could plausibly move, on your branch **and** on its merge base, and put both sets of numbers in the
-PR. Quote the `_core.so` md5 from each run's `meta` block: it is the only thing that identifies
-which binary produced a number.
+# Comprehensive Benchmarking
 
 ## Which group
 
@@ -37,11 +28,6 @@ propagators.
 Run one group of operations per process at scale: `build_graph`/`propagate` and `energy`/`gradient`
 each hold their own operator, and selecting all four holds two per rank.
 
-**Terms are exact; time and memory are not.** A term count is a property of the flags alone and is
-identical at every rank and partition count (`cpp/tests/partition_equivalence_tests.cpp`) — so it
-transfers to your machine and a disagreement means the *problem* changed. The `~s` and `~GiB/node`
-columns are one measurement on 128-core, 242 GiB nodes, rounded to a figure you can size a job
-with. They are not a baseline; your baseline is the merge base measured next to your branch.
 
 ## L1 — one thread, ~10M terms
 
@@ -63,29 +49,12 @@ they are where the operations separate.
 | --- | --- | --- | ---: | ---: | ---: |
 | hubbard `propagate` | `--hubbard-cutoff=10 --hubbard-lower-atol=3.38e-06` | `test_model_propagate and hubbard` | 1,001,661,534 | 140 | 60 |
 | pauli `propagate` | `--pauli-cutoff=14 --pauli-lower-atol=8.9e-06` | `test_model_propagate and pauli` | 985,970,588 | 90 | 80 |
-| random `propagate` | *(random flags below)* | `test_random_propagate and heisenberg` | 948,937,993 | 50 | 95 |
+| random `propagate` | `--num-generators=1000 --num-modes=142 --cutoff=6 --obs-terms=14750000` | `test_random_propagate and heisenberg` | 948,937,993 | 50 | 95 |
 | random `build_graph` | *(same)* | `test_random_build_graph and heisenberg` | 948,937,993 | 50 | 110 |
 | random `energy`+`gradient` | *(same)* | `(test_random_energy or test_random_gradient) and heisenberg` | 948,937,993 | 40 | 110 |
 | Schrödinger `build_graph` | *(same)* | `test_random_build_graph and schrodinger` | 18,825,440 | 1.5 | 17 |
 | Schrödinger `energy`+`gradient` | *(same)* | `(test_random_energy or test_random_gradient) and schrodinger` | 18,825,440 | 0.3 | 11 |
 
-The random flags, identical on every random row:
-
-```
---num-generators=1000 --num-modes=142 --cutoff=6 --obs-terms=14750000
-```
-
-`--obs-terms` is the size knob in the Heisenberg picture, and at 1000 generators it multiplies by
-about 64 on the way to the propagated operator (14.75M observable terms → 948,937,993) — so walk
-**up** from a small observable, because an overshoot is an OOM rather than a slow run. It is
-**not** a size knob in the Schrödinger picture, which evolves the state: 200k, 2M and 14.75M
-observable terms all give ~19M. Nor can the other axes compensate — `--cutoff 8` and
-`--num-modes 200` each exceeded a 242 GiB node. ~19M is the picture's ceiling, measured.
-
-At ~949M terms on one rank over 128 cores the four operations cost 0.100, 0.107, 0.115 and 0.115
-GiB per Mterm (`propagate`, `energy`, `build_graph`, `gradient`) — within 15% of each other, so no
-operation is a multiple of another.
-
 ## L3 — four nodes, the same problems
 
 Same flags, same term counts, same cores per node as L2; 8 ranks per node × 16 partitions instead
@@ -102,22 +71,13 @@ having both.
 | Schrödinger `build_graph` | 1.1 | 80 | 1.3x faster, **4.8x more memory** |
 | Schrödinger `energy`+`gradient` | 0.15 | 40 | 2.0x faster, **3.5x more memory** |
 
-Time partitions across nodes; memory does not always. The cause is a **~9.5 GiB per-process
-resident floor**, paid once per rank, so eight ranks on a node pay it eight times: on random
-`build_graph` that is 75.6 of the measured 104.2 GiB/node before the build allocates anything.
-More ranks per node buys speed and costs a floor, and it hurts most where the operation's own cost
-is smallest — which is why the 19M-term Schrödinger rows cost more per node on four nodes than on
-one. The benchmarks page has the per-rank breakdown.
 
 ## L4 — strong and weak scaling
 
 Hubbard `propagate` only: a communication cost is a function of `P`, so varying the model as well
 would confound it. 8 ranks per node × 16 partitions throughout, so `R` = 8N and `P` = 128N.
 
-Size is set by `--hubbard-lower-atol` at a fixed `--hubbard-cutoff=10`. The cutoff is saturated
-there — at `atol=1.25e-05`, cutoffs 10/12/14 give 96.98M/105.79M/106.82M terms — so all growth
-comes from the tolerance and every point stays inside the same operator family. The sequence
-doubles, which is what lets a cell be shared by both ladders:
+Size is set by `--hubbard-lower-atol` at a fixed `--hubbard-cutoff=10`. 
 
 | k | `--hubbard-lower-atol` | terms |
 | ---: | --- | ---: |
@@ -133,9 +93,6 @@ doubles, which is what lets a cell be shared by both ladders:
 | 9 | `2.94e-07` | 48,317,129,677 |
 | 10 | `1.82e-07` | 94,684,031,363 |
 
-Above k=6 every size was calibrated by measurement: the local exponent
-`d(log terms)/d(-log atol)` drifts from 1.83 at k=1 to 1.40 at k=10, so a power law fitted at one
-end does not predict the other.
 
 **Weak** — hold terms per node constant: **step one row down the table each time you double the
 node count.** Three ladders, each seven points from N=1 to N=64:
@@ -159,25 +116,12 @@ That spread **is** the weak-scaling result; ideal scaling would be flat. Efficie
 
 Efficiency against the ladder's smallest node count `N0` is `N0 · t(N0) / (N · t(N))`. The larger
 two ladders start above one node because the problem does not fit below it, not because the point
-was skipped. Size a point with
-
-```
-GiB/node ≈ 3.4 + 0.063 × Mterms/node
-```
-
-fitted over the campaign's 38 points and specific to hubbard `propagate` at 8 ranks × 16
-partitions — a 242 GiB node holds about 3000 Mterms/node. The law does not transfer to another
-model: the random model measures 0.100–0.115 GiB/Mterm against this slope's 0.063.
-
-The measured answer already exists on the benchmarks page, so a new campaign is a diff against it
-rather than a first measurement. Its headline: each doubling of the problem buys one more useful
-doubling of nodes, and a run that can afford ~1.5e9 terms per node weak-scales at 84% to 8192
-cores while one at ~100M/node reaches 29%.
+was skipped. 
 
 ## Shapes
 
 Three numbers: nodes `N`, ranks per node `R`, partitions per rank `P`, one core per partition —
-so `R × P` is the cores per node. `P` is an environment variable, not a pytest flag.
+so `R × P` is the cores per node.
 
 One rank over a whole node — L1 (with `P`=1) and L2:
 
@@ -220,21 +164,12 @@ Each of these produces a plausible wrong number rather than an error.
   core.** Measured in one allocation: no flags → 128 CPUs, `--cpus-per-task=128` → 128,
   `--cpu-bind=cores` alone → 1. Repeat both flags on the `srun`, not only on the `#SBATCH`.
 - **Export `monoprop_PARTITIONS`.** The engine's own default is `ranks == 1 ? cores : 1`, so an
-  unset knob on a multi-rank run measures one partition per rank at a wall time plausible enough to
-  publish.
+  unset knob on a multi-rank run measures one partition per rank.
 - **`--cpu-bind=none` cost 1.45x** against Slurm's own cgroup pinning at 8 ranks per node. With one
   rank holding a whole node there is nothing to confine and it stops mattering.
-- **Leave `MALLOC_ARENA_MAX` unset.** Setting it to the partition count cost ~16% of wall.
 - **`nproc` lies inside a job** — it honours `OMP_NUM_THREADS`, which the BLAS pinning sets to 1,
   so a job holding 128 CPUs can log one core. Read `os.sched_getaffinity(0)`.
 - **Peak RSS follows capacity, not live bytes**, so a `shrink_to_fit` can raise it: `realloc` holds
   the old and the new buffer at once.
 - **`--hubbard-observable-site` (46) must stay below `--hubbard-num-sites` (60).** Shrinking the
   model without moving the observable fails.
-
-## When a term count moves
-
-Nothing here refuses a run, so read the term count yourself. A count that differs from this file is
-an **accuracy change**, and it is the only signal that a timing win was not one. Say so in the PR,
-and update the row in the same PR. Recalibrating a row costs one cheap run: the term count is the
-same at every shape, so measure it at the smallest one.

From 261bdb9723f863bf423ddfa862094b13534420d9 Mon Sep 17 00:00:00 2001
From: Aaron Miller <61472721+diagonal-hamiltonian@users.noreply.github.com>
Date: Mon, 31 Aug 2026 22:15:52 +0100
Subject: [PATCH 03/14] Revise benchmarks documentation and add ladder section

Updated the benchmarks documentation to include a new section on the benchmarking ladder and improved clarity on scaling and memory usage.

Signed-off-by: Aaron Miller <61472721+diagonal-hamiltonian@users.noreply.github.com>
---
 docs/content/docs/benchmarks.mdx | 156 +------------------------------
 1 file changed, 2 insertions(+), 154 deletions(-)

diff --git a/docs/content/docs/benchmarks.mdx b/docs/content/docs/benchmarks.mdx
index 7e910874..e34f9380 100644
--- a/docs/content/docs/benchmarks.mdx
+++ b/docs/content/docs/benchmarks.mdx
@@ -299,160 +299,8 @@ just bench-build-mpi                           # build once (MPI on)
 monoprop_NUM_THREADS=2 just bench-mpi r5t2 5 --map-by slot:PE=2 --bind-to core
 ```
 
-### The ladder
-
-The default sizes above fit a laptop and a CI runner; the sizes the library is used at do not.
-`benches/LADDER.md` is the set at those sizes, run through the ordinary
-`just bench LABEL <flags>` route (or that same pytest line under `srun` for multi-rank): four
-groups, each a table of the exact flags and `-k` selector that produce it. L1 is one thread at
-~10M terms; L2 is one node at ~1B; L3 is the same problems on four nodes at 8 ranks/node x 16
-partitions; L4 is the strong and weak scaling ladders of hubbard `propagate`. The document also
-gives the sbatch shapes for each group and the pinning traps that silently unpin a run.
-
-Nothing runs these for you, and no benchmark gates a pull request. Running the group a change
-could plausibly move, and putting the numbers in the pull request, is the author's job.
-
-### Scaling
-
-Node scaling of Hubbard `propagate`, 1 to 64 nodes (128 to
-8192 cores) at 8 MPI ranks per node × 16 partitions, so `R = 8N` and `P = 128N` with one core per
-partition. Measured 2026-08-27 on AMD EPYC 7742 nodes with 242.0 GiB usable each; 38 points, five
-reps apiece (ten on the smallest weak ladder), every rep gate-checked on binary identity,
-environment and term count before its time was kept. `benches/LADDER.md`'s L4 group reproduces
-these points; the medians below are what a run of it is a diff against.
-
-The measuring binary was `perf/linear-routing-on-wire` @ `c2554554` (`_core.so` md5
-`1157a5e2b421fb8bd18fc6d16fa39778`), whose default is GF(2)-linear on-wire routing. That is the
-branch's default, **not** `main`'s: the figures are internally consistent and traceable, but they
-are not a measurement of `main`.
-
-Problem size is `--hubbard-lower-atol` at a fixed cutoff of 10, which is saturated there, so all
-growth comes from the tolerance and every point stays inside the same operator family. The eleven
-sizes form a doubling sequence and both families are built from it, so a cell reached by two
-ladders is *exactly* the same problem — nine such cells agreed to within 1.24%, most within 0.2%,
-which is a free check on the whole apparatus.
-
-Some points are absent because the problem does not fit in that many nodes' memory, not because
-they were skipped: `strong-6126m` starts at two nodes and `strong-24420m` at eight.
-
-**`strong-1569m`** — 1,569,152,761 terms held fixed.
-
-| nodes | R | P | terms | Mterms/node | median s | Mterms/s/node | GiB/node |
-|---:|---:|---:|---:|---:|---:|---:|---:|
-| 1 | 8 | 128 | 1,569,152,761 | 1569 | 215.61 | 7.28 | 99.9 |
-| 2 | 16 | 256 | 1,569,152,761 | 785 | 105.91 | 7.41 | 51.5 |
-| 4 | 32 | 512 | 1,569,152,761 | 392 | 53.34 | 7.35 | 28.0 |
-| 8 | 64 | 1024 | 1,569,152,761 | 196 | 28.58 | 6.86 | 15.9 |
-| 16 | 128 | 2048 | 1,569,152,761 | 98 | 18.12 | 5.41 | 11.0 |
-| 32 | 256 | 4096 | 1,569,152,761 | 49 | 16.13 | 3.04 | 6.6 |
-| 64 | 512 | 8192 | 1,569,152,761 | 25 | 23.99 | 1.02 | 4.1 |
-
-**`strong-6126m`** — 6,125,805,627 terms held fixed.
-
-| nodes | R | P | terms | Mterms/node | median s | Mterms/s/node | GiB/node |
-|---:|---:|---:|---:|---:|---:|---:|---:|
-| 2 | 16 | 256 | 6,125,805,627 | 3063 | 450.19 | 6.80 | 200.2 |
-| 4 | 32 | 512 | 6,125,805,627 | 1531 | 224.45 | 6.82 | 101.3 |
-| 8 | 64 | 1024 | 6,125,805,627 | 766 | 109.53 | 6.99 | 51.3 |
-| 16 | 128 | 2048 | 6,125,805,627 | 383 | 58.75 | 6.52 | 27.4 |
-| 32 | 256 | 4096 | 6,125,805,627 | 191 | 38.09 | 5.03 | 16.1 |
-| 64 | 512 | 8192 | 6,125,805,627 | 96 | 36.95 | 2.59 | 11.4 |
-
-**`strong-24420m`** — 24,419,998,198 terms held fixed.
-
-| nodes | R | P | terms | Mterms/node | median s | Mterms/s/node | GiB/node |
-|---:|---:|---:|---:|---:|---:|---:|---:|
-| 8 | 64 | 1024 | 24,419,998,198 | 3052 | 471.03 | 6.48 | 201.3 |
-| 16 | 128 | 2048 | 24,419,998,198 | 1526 | 235.19 | 6.49 | 101.8 |
-| 32 | 256 | 4096 | 24,419,998,198 | 763 | 127.10 | 6.00 | 52.2 |
-| 64 | 512 | 8192 | 24,419,998,198 | 382 | 79.58 | 4.79 | 26.9 |
-
-**`weak-97m`** — ~97M terms per node.
-
-| nodes | R | P | terms | Mterms/node | median s | Mterms/s/node | GiB/node |
-|---:|---:|---:|---:|---:|---:|---:|---:|
-| 1 | 8 | 128 | 96,981,051 | 97 | 10.75 | 9.02 | 10.8 |
-| 2 | 16 | 256 | 184,124,520 | 92 | 10.58 | 8.70 | 9.6 |
-| 4 | 32 | 512 | 377,482,074 | 94 | 13.13 | 7.19 | 11.5 |
-| 8 | 64 | 1024 | 781,669,404 | 98 | 14.54 | 6.72 | 10.9 |
-| 16 | 128 | 2048 | 1,569,152,761 | 98 | 18.11 | 5.42 | 11.0 |
-| 32 | 256 | 4096 | 3,104,527,573 | 97 | 24.39 | 3.98 | 11.0 |
-| 64 | 512 | 8192 | 6,125,805,627 | 96 | 36.59 | 2.62 | 11.4 |
-
-**`weak-385m`** — ~385M terms per node.
-
-| nodes | R | P | terms | Mterms/node | median s | Mterms/s/node | GiB/node |
-|---:|---:|---:|---:|---:|---:|---:|---:|
-| 1 | 8 | 128 | 377,482,074 | 377 | 47.71 | 7.91 | 26.6 |
-| 2 | 16 | 256 | 781,669,404 | 391 | 50.90 | 7.68 | 28.0 |
-| 4 | 32 | 512 | 1,569,152,761 | 392 | 53.29 | 7.36 | 27.9 |
-| 8 | 64 | 1024 | 3,104,527,573 | 388 | 54.73 | 7.09 | 27.5 |
-| 16 | 128 | 2048 | 6,125,805,627 | 383 | 58.03 | 6.60 | 27.4 |
-| 32 | 256 | 4096 | 12,255,330,837 | 383 | 64.64 | 5.92 | 26.3 |
-| 64 | 512 | 8192 | 24,419,998,198 | 382 | 79.89 | 4.78 | 26.9 |
-
-**`weak-1529m`** — ~1529M terms per node.
-
-| nodes | R | P | terms | Mterms/node | median s | Mterms/s/node | GiB/node |
-|---:|---:|---:|---:|---:|---:|---:|---:|
-| 1 | 8 | 128 | 1,569,152,761 | 1569 | 215.61 | 7.28 | 99.9 |
-| 2 | 16 | 256 | 3,104,527,573 | 1552 | 222.27 | 6.98 | 100.2 |
-| 4 | 32 | 512 | 6,125,805,627 | 1531 | 224.04 | 6.84 | 101.2 |
-| 8 | 64 | 1024 | 12,255,330,837 | 1532 | 227.33 | 6.74 | 99.9 |
-| 16 | 128 | 2048 | 24,419,998,198 | 1526 | 235.55 | 6.48 | 101.8 |
-| 32 | 256 | 4096 | 48,317,129,677 | 1510 | 249.62 | 6.05 | 101.3 |
-| 64 | 512 | 8192 | 94,684,031,363 | 1479 | 257.05 | 5.76 | 84.3 |
-
-
-Two things follow from the strong curves, and only from having three of them. The reversal in
-`strong-1569m` at 64 nodes is not a property of a core count: the same hardware still improves at
-64 nodes on the two larger problems. What sets the optimum is the load *per node*, and it falls
-below roughly 50–100 Mterms/node at the point each curve turns. Each doubling of the problem buys
-one more useful doubling of nodes.
-
-Weak scaling says the same thing from the other side. At ~97M terms/node, 64 nodes runs at 29% of
-one node's efficiency; at ~385M it is 60%; at ~1529M it is 84% across 64× the hardware. A run that
-can afford about 1.5e9 terms per node weak-scales almost ideally to 8192 cores; one at 100M/node
-does not.
-
-Peak resident memory over all 38 points, spanning 25 to 3063 Mterms per node:
-
-```
-GiB/node = 3.39 + 0.0634 × Mterms/node
-```
-
-The law is **specific to hubbard `propagate` at 8 ranks x 16 partitions** and does not transfer:
-the random model at ~949M terms measures 0.100 GiB/Mterm propagating, 0.115 building the graph
-and 0.115 evaluating energy and gradient, against this slope's 0.063. Both terms of the fit are
-real. The slope is the per-term cost, about 68 bytes per term. The constant is a
-genuine per-node floor that a model through the origin cannot reproduce — the smallest point
-measures 4.1 GiB/node at 25 Mterms/node. Every point falls within −2.9 to +4.5 GiB/node of that
-line except `weak-1529m` at 64 nodes, which measures 84.3 against 97.1 predicted, identically on
-all five reps; it is reproducible and unexplained, and no mechanism is claimed for it on the
-strength of one point. The largest cell measured 201.3 GiB/node against a 242.0 GiB grant.
-
-Memory partitions across nodes for hubbard and pauli and **does not** for the random model, whose
-per-node peak barely moves on any operation. The cause is a per-process floor. For the random
-`build_graph` row of L2 against L3, `opmembase` is the resident floor when the timed call starts
-and `opmemdelta` is the call's own cost:
-
-| per rank | 1 node, R=1 | 4 nodes, R=32 |
-| --- | ---: | ---: |
-| floor (`opmembase`) | 13.04 GiB | 9.45 GiB |
-| the call itself (`opmemdelta`) | 96.25 GiB | 3.59 GiB |
-
-Both columns reconcile to the measured peak exactly: `13.0 + 96.3 = 109.3` GiB/node, and
-`8 x 9.45 + 8 x 3.59 = 104.2`. The operation's own memory partitions almost perfectly -- 27x less
-per rank for 32x the ranks -- while the floor is a **~9.5 GiB per-process constant**, so eight
-ranks on a node pay it eight times and 75.6 of that row's 104.2 GiB/node is floor before the graph
-build allocates anything. It is not per-partition (128 partitions carry 13.04 GiB, 16 carry 9.45)
-and it is not the operator: every field of `opmembreak` partitions cleanly, its rank-summed total
-moving only 88.0 to 86.0 GiB from 1 rank to 32. What the 9.5 GiB consists of is **not attributed**
-by any recorded field; allocator-retained construction memory, fabric buffers and per-process
-thread state are the unmeasured candidates. The consequence is the useful part: **more ranks per
-node buys speed and costs a floor**, paid per process, so it hurts most where the operation's own
-cost is smallest -- worst on the 19M-term Schrodinger rows, where four nodes cost more per node
-than one.
+### Benchmarking ladder.
+We include a `LADDER.md` file with information on large-scale MPI benchmarking for development work.
 
 ### Continuous benchmarking
 

From 9f5a353e4dfd59cb69957979330240356f93f30f Mon Sep 17 00:00:00 2001
From: Aaron Miller <aaron.miller@algorithmiq.fi>
Date: Mon, 31 Aug 2026 22:57:02 +0100
Subject: [PATCH 04/14] =?UTF-8?q?docs(benches):=20=F0=9F=93=9D=20general?=
 =?UTF-8?q?=20shapes,=20gradient=20rows,=20inputtable=20model=20sizes?=
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Addresses the four review comments on #317.

L1 gains two `gradient` rows at ~20M terms — one against the exact graph, one
against a plan pared at `1e-10` — so a change to the graph functionals shows on
the pair. `--pare-threshold` is a new pytest option threaded into
`expectation_value_functional` and its gradient twin for both the random and the
fixed-model benchmarks; unset reproduces today's behaviour exactly.

L3 and L4 no longer prescribe a shape. `N`, `R` and `P` are the caller's, the L3
table gives absolute ballparks like every other group instead of ratios against
L2, and declaring the shape is now mandatory rather than advisory: `conftest.py`
refuses a multi-rank session with `monoprop_PARTITIONS` unset, since the engine's
`ranks == 1 ? cores : 1` default would otherwise measure one partition per rank
at a plausible wall time. The shape is recorded on every run —
`partitions_env` unconditionally now, so "unset" and "not recorded" stop
rendering identically in `REPORT.md`.

The model size knobs become usable inputs. `--pauli-num-qubits` other than the
127 that `HEAVY_HEX_TOPOLOGY` couples, `--hubbard-observable-site` off its
lattice, and any mode count above the extension's `MAX_NUM_MODES` all raise
where they previously built a silently different model. `LADDER.md` documents
the knobs, their ranges, and that `--obs-terms` is an upper bound rather than an
exact count.

`bench-mpi` forwards `monoprop_PARTITIONS` and `monoprop_NUM_THREADS` through
`mpiexec`, which its own doc comment already assumed. AGENTS.md is cut to two
thirds, its truncated opening sentence repaired, and told to run `prek` before
pushing.

Assisted-by: claude-code:claude-opus-5
---
 benches/LADDER.md                             | 104 +++++++++++++-----
 benches/README.md                             |   5 +-
 benches/bench_models.py                       |  22 +++-
 benches/bench_random.py                       |  20 +++-
 benches/conftest.py                           |  53 ++++++---
 docs/content/docs/benchmarks.mdx              |  16 ++-
 justfile                                      |   9 +-
 .../src/monoprop_bench_tools/models.py        |  39 ++++++-
 8 files changed, 209 insertions(+), 59 deletions(-)

diff --git a/benches/LADDER.md b/benches/LADDER.md
index 100ff5cd..c93716bd 100644
--- a/benches/LADDER.md
+++ b/benches/LADDER.md
@@ -6,9 +6,10 @@
 | --- | --- |
 | a kernel, encoding or data structure | L1, then L2. Hubbard makes 29 `propagate` calls and pauli 1, so a per-call cost shows on one and hides on the other |
 | the graph build or contraction | L2's graph and eval rows, both pictures |
+| the graph functionals or their masked plan | L1's gradient rows, which run with and without `--pare-threshold` |
 | MPI, routing, exchange or placement | L4. A communication cost is a function of `P` = ranks × partitions, not of node count |
 | memory layout or allocation | L1 **and** L2. The per-term and the per-process terms of the memory cost only separate across sizes |
-| ranks per node, or anything paid per process | L2 against L3 — same cores per node, 1 rank against 8 |
+| ranks per node, or anything paid per process | L2 against L3 — same cores per node, one rank against several |
 
 ## Running one row
 
@@ -28,16 +29,56 @@ propagators.
 Run one group of operations per process at scale: `build_graph`/`propagate` and `energy`/`gradient`
 each hold their own operator, and selecting all four holds two per rank.
 
+## Declaring the shape
 
-## L1 — one thread, ~10M terms
+Three numbers describe every run: nodes `N`, ranks per node `R`, partitions per rank `P`, one core
+per partition — so `R × P` is the cores per node and `P_total` = `N × R × P` is what a
+communication cost is a function of. **`N` and `R` come from the launcher; `P` you must export.**
+
+```bash
+export monoprop_PARTITIONS=16 monoprop_NUM_THREADS=16
+```
+
+Above one rank this is mandatory: `benches/conftest.py` refuses to start a multi-rank session with
+`monoprop_PARTITIONS` unset, because the engine's own default is `ranks == 1 ? cores : 1` and an
+unset knob would silently measure one partition per rank at a plausible wall time.
+
+The shape is recorded with every run — `ranks`, `nodes`, `ranks_per_node`, `partitions_env` and
+`monoprop_threads` in `results/<label>.json`'s `meta`, surfaced as the Configuration table of
+`REPORT.md`. Check it before comparing two rows: a ladder whose points did not all run at the same
+`R` and `P` is not a ladder.
+
+## Model size knobs
+
+Every size below is an input, not a constant. The fixed models take `--<model>-<field>` for any
+field of their config dataclass (`just bench --help` lists them); the random problem takes
+`--num-generators`, `--num-modes`, `--cutoff`, `--obs-terms`, `--gen-length` and `--seed`.
+
+| knob | range | note |
+| --- | --- | --- |
+| `--num-modes` | ≤ this build's `MAX_NUM_MODES` | rejected above it, rather than failing inside the extension |
+| `--hubbard-num-sites` | 2 × sites ≤ `MAX_NUM_MODES` | saturated from 60 at the default `lower_atol`; size Hubbard with `--hubbard-lower-atol` |
+| `--hubbard-observable-site` | `< --hubbard-num-sites` | a lattice position, not a constant: the default 46 sits 46/60 along the lattice, and shrinking the lattice without moving it changes the light cone. Out of range now raises |
+| `--pauli-num-qubits` | 127 only | `HEAVY_HEX_TOPOLOGY` is the fixed IBM Eagle coupling map, so any other count either indexes past the operator or leaves qubits uncoupled. Raises. Size this model with `--pauli-lower-atol` |
+| `--obs-terms` | any | an upper bound on the observable's terms, not an exact count: monomials are drawn independently and duplicates collapse, by about `obs_terms / 2·C(2·num_modes, gen_length)` — 0.06% at L1's 295k, 2.8% at L2's 14.75M. Deterministic for a fixed `--seed` |
+| `--pare-threshold` | any, default off | edge-retention cutoff for the graph functionals (`energy`, `gradient`). Unset keeps the exact graph |
+
+## L1 — one thread
 
 One node, one rank, one partition: `export monoprop_PARTITIONS=1 monoprop_NUM_THREADS=1`.
 Removes every threading and MPI effect, so what moves is the kernel.
 
+The two `propagate` rows sit at ~10M terms and the two `gradient` rows at ~20M. The gradient rows
+are the same problem twice, once against the exact graph and once against a plan pared at
+`1e-10`, so a change to the functionals or to the masked plan shows on the pair rather than on
+either alone.
+
 | row | flags | `-k` | terms | ~s | ~GiB |
 | --- | --- | --- | ---: | ---: | ---: |
 | hubbard `propagate` | `--hubbard-cutoff=10 --hubbard-lower-atol=4.2e-05` | `test_model_propagate and hubbard` | 9,953,109 | 30 | 1 |
 | pauli `propagate` | `--pauli-cutoff=12 --pauli-lower-atol=1.22e-04` | `test_model_propagate and pauli` | 10,069,308 | 20 | 1 |
+| random `gradient` | `--num-generators=1000 --num-modes=142 --cutoff=6 --obs-terms=295000` | `test_random_gradient and heisenberg` | 19,902,244 | 11 | 2.5 |
+| random `gradient`, pared | *(same)* `--pare-threshold=1e-10` | `test_random_gradient and heisenberg` | 19,902,244 | 0.8 | 2.8 |
 
 ## L2 — one node, ~1B terms
 
@@ -55,29 +96,38 @@ they are where the operations separate.
 | Schrödinger `build_graph` | *(same)* | `test_random_build_graph and schrodinger` | 18,825,440 | 1.5 | 17 |
 | Schrödinger `energy`+`gradient` | *(same)* | `(test_random_energy or test_random_gradient) and schrodinger` | 18,825,440 | 0.3 | 11 |
 
-## L3 — four nodes, the same problems
+## L3 — several nodes, the same problems
 
-Same flags, same term counts, same cores per node as L2; 8 ranks per node × 16 partitions instead
-of 1 × 128, so `R` = 32 and `P` = 512. Directly comparable to L2 row for row, which is the point of
-having both.
+The L2 problems again, at your own `N`, `R` and `P`. Choose `R × P` equal to L2's core count and
+the pair isolates what is paid per process rather than per core; choose anything else and it is
+still a valid multi-node run, just not that comparison.
 
-| row | ~s | ~GiB/node | against L2 |
-| --- | ---: | ---: | --- |
-| hubbard `propagate` | 35 | 18 | 4.1x faster, 3.2x less memory |
-| pauli `propagate` | 23 | 25 | 4.1x faster, 3.3x less memory |
-| random `propagate` | 13 | 100 | 3.8x faster, **0.95x — slightly worse** |
-| random `build_graph` | 13 | 100 | 3.9x faster, **1.05x — flat** |
-| random `energy`+`gradient` | 9 | 60 | 4.3x faster, 1.8x less memory |
-| Schrödinger `build_graph` | 1.1 | 80 | 1.3x faster, **4.8x more memory** |
-| Schrödinger `energy`+`gradient` | 0.15 | 40 | 2.0x faster, **3.5x more memory** |
+Same flags and same `-k` as L2, so only the measured cells differ. Term counts are
+geometry-independent (`cpp/tests/partition_equivalence_tests.cpp`), which is why one calibration
+serves every shape and a term-count disagreement means the *problem* changed, not the topology.
 
+| row | terms | ~s | ~GiB/node |
+| --- | ---: | ---: | ---: |
+| hubbard `propagate` | 1,001,661,534 | 35 | 18 |
+| pauli `propagate` | 985,970,588 | 23 | 25 |
+| random `propagate` | 948,937,993 | 13 | 100 |
+| random `build_graph` | 948,937,993 | 13 | 100 |
+| random `energy`+`gradient` | 948,937,993 | 9 | 60 |
+| Schrödinger `build_graph` | 18,825,440 | 1.1 | 80 |
+| Schrödinger `energy`+`gradient` | 18,825,440 | 0.15 | 40 |
+
+Measured at `N`=4, `R`=8, `P`=16 (medians of two reps) — a ballpark for sizing a job at that
+shape, not a target. Memory per node does not fall the way time does: there is a per-process
+resident floor of roughly 9.5 GiB, paid once per rank whatever the problem, so a row whose
+operation is cheap in memory pays more per node at 8 ranks than at 1.
 
 ## L4 — strong and weak scaling
 
-Hubbard `propagate` only: a communication cost is a function of `P`, so varying the model as well
-would confound it. 8 ranks per node × 16 partitions throughout, so `R` = 8N and `P` = 128N.
+Hubbard `propagate` only: a communication cost is a function of `P_total`, so varying the model as
+well would confound it. Hold `R` and `P` fixed across every point of a ladder and vary only `N`;
+`P_total` is then `N × R × P`. The tables below were built at `R`=8, `P`=16.
 
-Size is set by `--hubbard-lower-atol` at a fixed `--hubbard-cutoff=10`. 
+Size is set by `--hubbard-lower-atol` at a fixed `--hubbard-cutoff=10`.
 
 | k | `--hubbard-lower-atol` | terms |
 | ---: | --- | ---: |
@@ -93,7 +143,6 @@ Size is set by `--hubbard-lower-atol` at a fixed `--hubbard-cutoff=10`.
 | 9 | `2.94e-07` | 48,317,129,677 |
 | 10 | `1.82e-07` | 94,684,031,363 |
 
-
 **Weak** — hold terms per node constant: **step one row down the table each time you double the
 node count.** Three ladders, each seven points from N=1 to N=64:
 
@@ -116,13 +165,10 @@ That spread **is** the weak-scaling result; ideal scaling would be flat. Efficie
 
 Efficiency against the ladder's smallest node count `N0` is `N0 · t(N0) / (N · t(N))`. The larger
 two ladders start above one node because the problem does not fit below it, not because the point
-was skipped. 
+was skipped.
 
 ## Shapes
 
-Three numbers: nodes `N`, ranks per node `R`, partitions per rank `P`, one core per partition —
-so `R × P` is the cores per node.
-
 One rank over a whole node — L1 (with `P`=1) and L2:
 
 ```bash
@@ -134,8 +180,8 @@ just bench L2-hubbard-branch --hubbard-cutoff=10 --hubbard-lower-atol=3.38e-06 \
     -k "test_model_propagate and hubbard"
 ```
 
-`N` nodes at 8 ranks × 16 partitions — L3 and L4. This is the same command with `srun` inserted
-and the report moved out of it:
+`N` nodes at `R` ranks × `P` partitions — L3 and L4. This is the same command with `srun` inserted
+and the report moved out of it; substitute your own `N`, `R` and `P` in all four places:
 
 ```bash
 #!/bin/bash
@@ -163,13 +209,13 @@ Each of these produces a plausible wrong number rather than an error.
 - **`srun --cpu-bind=cores` with no `--cpus-per-task` on the `srun` confines each task to one
   core.** Measured in one allocation: no flags → 128 CPUs, `--cpus-per-task=128` → 128,
   `--cpu-bind=cores` alone → 1. Repeat both flags on the `srun`, not only on the `#SBATCH`.
-- **Export `monoprop_PARTITIONS`.** The engine's own default is `ranks == 1 ? cores : 1`, so an
-  unset knob on a multi-rank run measures one partition per rank.
 - **`--cpu-bind=none` cost 1.45x** against Slurm's own cgroup pinning at 8 ranks per node. With one
   rank holding a whole node there is nothing to confine and it stops mattering.
 - **`nproc` lies inside a job** — it honours `OMP_NUM_THREADS`, which the BLAS pinning sets to 1,
   so a job holding 128 CPUs can log one core. Read `os.sched_getaffinity(0)`.
 - **Peak RSS follows capacity, not live bytes**, so a `shrink_to_fit` can raise it: `realloc` holds
   the old and the new buffer at once.
-- **`--hubbard-observable-site` (46) must stay below `--hubbard-num-sites` (60).** Shrinking the
-  model without moving the observable fails.
+- **A two-operation row's peak is the MAX over its operations, never the sum.** `HighWaterMark`
+  resets `VmHWM` per benchmark, so both windows contain the same resident operator.
+- Unset `monoprop_PARTITIONS` and out-of-range model sizes used to be traps of this kind. Both now
+  raise; see *Declaring the shape* and *Model size knobs*.
diff --git a/benches/README.md b/benches/README.md
index 852d1ac2..d5106c4e 100644
--- a/benches/README.md
+++ b/benches/README.md
@@ -42,7 +42,10 @@ roughly as `lower_atol^-1.9`.
 
 `--hubbard-observable-site` (default 46) is a lattice position, not a
 constant: sweeping `--hubbard-num-sites` without scaling it as 46/60 of the
-lattice slides the observable off-centre and changes the light cone.
+lattice slides the observable off-centre and changes the light cone. Off the
+lattice entirely it raises, as does a mode count above the extension's
+`MAX_NUM_MODES` and any `--pauli-num-qubits` other than the 127 that
+`HEAVY_HEX_TOPOLOGY` couples.
 
 `build_graph` extends the graph, so Hubbard's 29 Trotter steps retain 29
 layer-sets and exceed 229 GiB at 23.9M terms, where `propagate` runs the same
diff --git a/benches/bench_models.py b/benches/bench_models.py
index a5df09c5..8f21c4c9 100644
--- a/benches/bench_models.py
+++ b/benches/bench_models.py
@@ -130,12 +130,19 @@ def run(built, n_steps):
 @pytest.mark.slow
 @pytest.mark.parametrize("model", list(MODELS))
 def test_model_energy(
-    benchmark, model_graph, model, model_configs, bench_comm, bench_rounds, op_memory
+    benchmark,
+    model_graph,
+    model,
+    model_configs,
+    bench_comm,
+    bench_rounds,
+    op_memory,
+    pare_threshold,
 ):
     """Benchmark evaluating a fixed model's expectation-value functional."""
     skip_if_graph_will_not_fit(model, MODELS[model][2](model_configs[model]))
     propagator, parameters = model_graph(model)
-    functional = propagator.expectation_value_functional()
+    functional = propagator.expectation_value_functional(pare_threshold)
     op_memory.open()
     result = benchmark.pedantic(
         barriered(functional, bench_comm),
@@ -151,12 +158,19 @@ def test_model_energy(
 @pytest.mark.slow
 @pytest.mark.parametrize("model", list(MODELS))
 def test_model_gradient(
-    benchmark, model_graph, model, model_configs, bench_comm, bench_rounds, op_memory
+    benchmark,
+    model_graph,
+    model,
+    model_configs,
+    bench_comm,
+    bench_rounds,
+    op_memory,
+    pare_threshold,
 ):
     """Benchmark evaluating a fixed model's expectation-value-and-gradient functional."""
     skip_if_graph_will_not_fit(model, MODELS[model][2](model_configs[model]))
     propagator, parameters = model_graph(model)
-    functional = propagator.expectation_value_and_gradient_functional()
+    functional = propagator.expectation_value_and_gradient_functional(pare_threshold)
     op_memory.open()
     _value, gradient = benchmark.pedantic(
         barriered(functional, bench_comm),
diff --git a/benches/bench_random.py b/benches/bench_random.py
index 5c00b820..6a69843f 100644
--- a/benches/bench_random.py
+++ b/benches/bench_random.py
@@ -83,10 +83,16 @@ def run(built):
 
 
 def test_random_energy(
-    benchmark, built_graph, random_problem, bench_comm, bench_rounds, op_memory
+    benchmark,
+    built_graph,
+    random_problem,
+    bench_comm,
+    bench_rounds,
+    op_memory,
+    pare_threshold,
 ):
     """Benchmark evaluating the expectation-value functional."""
-    functional = built_graph.expectation_value_functional()
+    functional = built_graph.expectation_value_functional(pare_threshold)
     op_memory.open()
 
     # The entry barrier must stay untimed, so it lives in a setup, and a setup that returns args forbids args=.
@@ -104,10 +110,16 @@ def setup():
 
 
 def test_random_gradient(
-    benchmark, built_graph, random_problem, bench_comm, bench_rounds, op_memory
+    benchmark,
+    built_graph,
+    random_problem,
+    bench_comm,
+    bench_rounds,
+    op_memory,
+    pare_threshold,
 ):
     """Benchmark evaluating the expectation-value-and-gradient functional."""
-    functional = built_graph.expectation_value_and_gradient_functional()
+    functional = built_graph.expectation_value_and_gradient_functional(pare_threshold)
     op_memory.open()
 
     def setup():
diff --git a/benches/conftest.py b/benches/conftest.py
index 8c44bb1d..0ebb954c 100644
--- a/benches/conftest.py
+++ b/benches/conftest.py
@@ -123,14 +123,17 @@ def _spread(comm: Any, value: int) -> dict[str, int]:
     return {"sum": _reduce_sum(comm, value), "max": _reduce_max(comm, value)}
 
 
+# (name, type, default, help). The type is per row because ``--pare-threshold`` is a float
+# that defaults to off, where every size knob is an int.
 _RANDOM_OPTIONS = (
-    ("gen-length", 4, "Majorana operators per generator."),
-    ("obs-terms", 10000, "Observable terms."),
-    ("num-generators", 100, "Random generators (circuit gates)."),
-    ("num-modes", 128, "Fermionic modes."),
-    ("cutoff", 6, "Truncation cutoff."),
-    ("seed", 0, "Random seed."),
-    ("bench-rounds", 1, "Fixed timing rounds (MPI-safe)."),
+    ("gen-length", int, 4, "Majorana operators per generator."),
+    ("obs-terms", int, 10000, "Observable terms."),
+    ("num-generators", int, 100, "Random generators (circuit gates)."),
+    ("num-modes", int, 128, "Fermionic modes."),
+    ("cutoff", int, 6, "Truncation cutoff."),
+    ("seed", int, 0, "Random seed."),
+    ("bench-rounds", int, 1, "Fixed timing rounds (MPI-safe)."),
+    ("pare-threshold", float, None, "Edge-retention cutoff for the graph functionals."),
 )
 
 _RESULTS: dict[str, Any] = {
@@ -170,8 +173,8 @@ def _results_path() -> Path | None:
 def pytest_addoption(parser: pytest.Parser) -> None:
     """Register benchmark configuration options for the random benchmarks."""
     group = parser.getgroup("monoprop-bench", "monoprop random benchmark sizing")
-    for name, default, help_text in _RANDOM_OPTIONS:
-        group.addoption(f"--{name}", type=int, default=default, help=help_text)
+    for name, kind, default, help_text in _RANDOM_OPTIONS:
+        group.addoption(f"--{name}", type=kind, default=default, help=help_text)
 
     models = parser.getgroup("monoprop-models", "monoprop fixed-model overrides")
     for model, (config_cls, _builder, _steps) in MODELS.items():
@@ -228,9 +231,8 @@ def _meta(nodes: int, ranks_per_node: int) -> dict[str, Any]:
     }
     # No nanobind property exposes the engine's resolved partition count (see bindings/binder.h),
     # so record the requested env var under its own name rather than claim it is the effective value.
-    partitions_env = os.environ.get("monoprop_PARTITIONS")  # noqa: SIM112
-    if partitions_env is not None:
-        meta["partitions_env"] = partitions_env
+    # Recorded even when absent: "unset" and a missing field render identically in the report.
+    meta["partitions_env"] = os.environ.get("monoprop_PARTITIONS", "unset")  # noqa: SIM112
     return meta
 
 
@@ -238,10 +240,28 @@ def _params(config: pytest.Config) -> dict[str, Any]:
     """Return the resolved random-problem hyperparameters (defaults included)."""
     return {
         name.replace("-", "_"): config.getoption(f"--{name}")
-        for name, _default, _help in _RANDOM_OPTIONS
+        for name, _kind, _default, _help in _RANDOM_OPTIONS
     }
 
 
+def _require_shape() -> None:
+    """Fail a multi-rank run that has not declared its partition count.
+
+    ``resolve_partition_count_`` defaults to ``ranks == 1 ? cores : 1``, so an unset knob
+    under MPI measures one partition per rank -- a single-threaded run at a plausible wall
+    time, with nothing in the timing to say so. Every rank raises, so no rank is left in a
+    collective. Serial runs take the engine default and are unaffected.
+    """
+    if _size() > 1 and not os.environ.get("monoprop_PARTITIONS"):  # noqa: SIM112
+        msg = (
+            "monoprop_PARTITIONS is unset on a run of "
+            f"{_size()} ranks. The engine would use one partition per rank. Export the "
+            "shape you intend, e.g. `export monoprop_PARTITIONS=16 monoprop_NUM_THREADS=16` "
+            "for 8 ranks per 128-core node."
+        )
+        raise pytest.UsageError(msg)
+
+
 @pytest.hookimpl(trylast=True)
 def pytest_configure(config: pytest.Config) -> None:
     """Record run metadata on rank 0; silence the other ranks under MPI.
@@ -250,6 +270,7 @@ def pytest_configure(config: pytest.Config) -> None:
     ``trylast`` so the terminal reporter exists before non-root ranks unregister it.
     """
     nodes, ranks_per_node = _nodes()  # collective; every rank must call this
+    _require_shape()
     if _rank() == 0:
         _RESULTS["meta"] = _meta(nodes, ranks_per_node)
         _RESULTS["params"] = _params(config)
@@ -291,6 +312,12 @@ def bench_rounds(request: pytest.FixtureRequest) -> int:
     return int(request.config.getoption("--bench-rounds"))
 
 
+@pytest.fixture(scope="session")
+def pare_threshold(request: pytest.FixtureRequest) -> float | None:
+    """Return the edge-retention cutoff for the graph functionals (``None`` keeps the exact graph)."""
+    return request.config.getoption("--pare-threshold")
+
+
 @pytest.fixture(scope="session")
 def model_configs(request: pytest.FixtureRequest) -> dict[str, Any]:
     """Return each fixed model's config, every field resolved from the CLI.
diff --git a/docs/content/docs/benchmarks.mdx b/docs/content/docs/benchmarks.mdx
index e34f9380..859185ca 100644
--- a/docs/content/docs/benchmarks.mdx
+++ b/docs/content/docs/benchmarks.mdx
@@ -294,13 +294,21 @@ across ranks. Memory is the sum of the per-rank peak RSS values. This is an uppe
 job-wide peak: shared pages are counted once per rank, and rank-local peaks are summed even when
 they occur at different times.
 
+Above one rank, `monoprop_PARTITIONS` is mandatory: the engine's own default is
+`ranks == 1 ? cores : 1`, so an unset knob would silently measure one partition per rank, and the
+benchmark session refuses to start without it. The shape it declares is recorded in each run's
+`meta` and shown in the report's configuration table.
+
 ```bash
 just bench-build-mpi                           # build once (MPI on)
-monoprop_NUM_THREADS=2 just bench-mpi r5t2 5 --map-by slot:PE=2 --bind-to core
+monoprop_PARTITIONS=2 monoprop_NUM_THREADS=2 just bench-mpi r5t2 5 --map-by slot:PE=2 --bind-to core
 ```
 
-### Benchmarking ladder.
-We include a `LADDER.md` file with information on large-scale MPI benchmarking for development work.
+### Benchmarking ladder
+
+`benches/LADDER.md` holds the ladder used for large-scale MPI benchmarking during development: four
+groups of configurations, the flags and `-k` selector that produce each row, and the shapes to run
+them at.
 
 ### Continuous benchmarking
 
@@ -344,7 +352,7 @@ runner.
 
 **No benchmark workflow gates a pull request.** Both track the main branch, and a threshold
 over a series that spans runners is a report, not a verdict. Measuring a change is the author's
-job -- see "The ladder" above. Every run uploads `bmf.json` and the raw result JSON as an artifact,
+job -- see "Benchmarking ladder" above. Every run uploads `bmf.json` and the raw result JSON as an artifact,
 so a failed Bencher upload does not take the measurement with it.
 
 The `bench-markers` and `bench-rounds` inputs of the reusable `bench.yml` workflow select the
diff --git a/justfile b/justfile
index da08ee19..b0f2055a 100644
--- a/justfile
+++ b/justfile
@@ -349,9 +349,10 @@ bench LABEL *ARGS:
     uv run --no-sync monoprop-bench-report "{{ bench_results }}"
 
 # Needs an MPI build (`just bench-build-mpi`) -- a non-MPI build is rejected by the
-# preflight. Extra args are passed to mpiexec for pinning (and, as root, add
+# preflight. `monoprop_PARTITIONS` is mandatory above one rank (benches/conftest.py refuses
+# the run without it). Extra args are passed to mpiexec for pinning (and, as root, add
 # `--allow-run-as-root`), e.g.
-#   monoprop_NUM_THREADS=2 just bench-mpi r5t2 5 --map-by slot:PE=2 --bind-to core
+#   monoprop_PARTITIONS=2 monoprop_NUM_THREADS=2 just bench-mpi r5t2 5 --map-by slot:PE=2 --bind-to core
 
 # Run under MPI: RANKS ranks recorded as one LABEL column.
 bench-mpi LABEL RANKS *MPIARGS:
@@ -360,7 +361,9 @@ bench-mpi LABEL RANKS *MPIARGS:
     label="$1"; ranks="$2"; shift 2; \
     monoprop_BENCH_LABEL="$label" monoprop_BENCH_RESULTS="{{ bench_results }}" \
         uv run --no-sync mpiexec -n "$ranks" \
-        -x monoprop_BENCH_LABEL -x monoprop_BENCH_RESULTS "$@" \
+        -x monoprop_BENCH_LABEL -x monoprop_BENCH_RESULTS \
+        ${monoprop_PARTITIONS+-x monoprop_PARTITIONS} \
+        ${monoprop_NUM_THREADS+-x monoprop_NUM_THREADS} "$@" \
         python -m pytest benches -o filterwarnings=default \
         --benchmark-json="{{ bench_results }}/time-$label.json"
     uv run --no-sync monoprop-bench-report "{{ bench_results }}"
diff --git a/packages/monoprop-bench-tools/src/monoprop_bench_tools/models.py b/packages/monoprop-bench-tools/src/monoprop_bench_tools/models.py
index 1c9ae567..bd4bd440 100644
--- a/packages/monoprop-bench-tools/src/monoprop_bench_tools/models.py
+++ b/packages/monoprop-bench-tools/src/monoprop_bench_tools/models.py
@@ -28,6 +28,7 @@
 import numpy as np
 
 from monoprop import (
+    MAX_NUM_MODES,
     Circuit,
     ExpGate,
     MajoranaPropagator,
@@ -120,6 +121,16 @@ def parameters(self) -> np.ndarray:
         return np.asarray(self.circuit.parameters, dtype=float)
 
 
+def _check_modes(num_modes: int, option: str) -> None:
+    """Reject a size above the extension's compile-time mode limit."""
+    if num_modes > MAX_NUM_MODES:
+        msg = (
+            f"{option} gives {num_modes} modes, above this build's MAX_NUM_MODES "
+            f"({MAX_NUM_MODES}). Rebuild with -Dmonoprop_MAX_NUM_MODES to go higher."
+        )
+        raise ValueError(msg)
+
+
 def _random_terms(
     rng: np.random.Generator,
     num_terms: int,
@@ -177,6 +188,7 @@ def make_random_problem(
     Returns:
         A :class:`RandomProblem` bundling the observable, circuit, and cutoff.
     """
+    _check_modes(num_modes, "--num-modes")
     rng = np.random.default_rng(seed)
     num_majorana_indices = 2 * num_modes
 
@@ -321,6 +333,15 @@ def build_hubbard_problem(
         ``(propagator, circuit)`` for one Trotter step, re-applied by the driver.
     """
     config = config or HubbardConfig()
+    _check_modes(config.num_qubits, "--hubbard-num-sites")
+    # A lattice position, not a constant: off the lattice it silently changes the light cone.
+    if not 0 <= config.observable_site < config.num_sites:
+        msg = (
+            f"--hubbard-observable-site {config.observable_site} is outside the "
+            f"{config.num_sites}-site lattice. Scale it with --hubbard-num-sites "
+            "(the default 46 sits 46/60 along the lattice)."
+        )
+        raise ValueError(msg)
     fermi_gates = [ExpGate(term) for term in _hubbard_fermion_terms(config)]
     parameters = [config.trotter_dt] * len(fermi_gates)
     occupied = _neel_occupied_modes(config.num_sites, config.neel_start_spin)
@@ -501,12 +522,15 @@ def build_hubbard_problem(
     (125, 126),
 ]
 
+# The map is fixed, so it also fixes the register: qubit counts are checked against it.
+HEAVY_HEX_NUM_QUBITS: int = 1 + max(j for _, j in HEAVY_HEX_TOPOLOGY)
+
 
 @dataclass(frozen=True, slots=True)
 class KickedIsingConfig:
     """Configuration for the static Pauli-basis (kicked-Ising) benchmark."""
 
-    num_qubits: int = 127
+    num_qubits: int = HEAVY_HEX_NUM_QUBITS
     num_layers: int = 20
     observable_qubit: int = 62
     theta: float = np.pi / 4
@@ -551,6 +575,19 @@ def build_kicked_ising_problem(
         ``(propagator, circuit)`` ready for in-place propagation.
     """
     config = config or KickedIsingConfig()
+    if config.num_qubits != HEAVY_HEX_NUM_QUBITS:
+        msg = (
+            f"--pauli-num-qubits must be {HEAVY_HEX_NUM_QUBITS}: HEAVY_HEX_TOPOLOGY is the "
+            "fixed IBM Eagle coupling map, so any other count either indexes past the "
+            "operator or leaves qubits uncoupled. Size this model with --pauli-lower-atol."
+        )
+        raise ValueError(msg)
+    if not 0 <= config.observable_qubit < config.num_qubits:
+        msg = (
+            f"--pauli-observable-qubit {config.observable_qubit} is outside the "
+            f"{config.num_qubits}-qubit register."
+        )
+        raise ValueError(msg)
     gate_angles: list[tuple[ExpGate, float]] = []
     for _ in range(config.num_layers):
         gate_angles.extend(_xlayer(config.num_qubits, config.theta / 2))

From b4a002948be72a0bdc0c5432eefe0a5909b0895b Mon Sep 17 00:00:00 2001
From: Aaron Miller <61472721+diagonal-hamiltonian@users.noreply.github.com>
Date: Tue, 1 Sep 2026 08:36:45 +0100
Subject: [PATCH 05/14] Revise LADDER.md for better benchmarking summary

Updated the LADDER.md file to improve clarity and structure of benchmarking information.

Signed-off-by: Aaron Miller <61472721+diagonal-hamiltonian@users.noreply.github.com>
---
 benches/LADDER.md | 16 +++++++---------
 1 file changed, 7 insertions(+), 9 deletions(-)

diff --git a/benches/LADDER.md b/benches/LADDER.md
index c93716bd..e64228ff 100644
--- a/benches/LADDER.md
+++ b/benches/LADDER.md
@@ -1,15 +1,13 @@
 # Comprehensive Benchmarking
 
-## Which group
+## Summary
 
-| a change to | run |
+| Group | What it measures |
 | --- | --- |
-| a kernel, encoding or data structure | L1, then L2. Hubbard makes 29 `propagate` calls and pauli 1, so a per-call cost shows on one and hides on the other |
-| the graph build or contraction | L2's graph and eval rows, both pictures |
-| the graph functionals or their masked plan | L1's gradient rows, which run with and without `--pare-threshold` |
-| MPI, routing, exchange or placement | L4. A communication cost is a function of `P` = ranks × partitions, not of node count |
-| memory layout or allocation | L1 **and** L2. The per-term and the per-process terms of the memory cost only separate across sizes |
-| ranks per node, or anything paid per process | L2 against L3 — same cores per node, one rank against several |
+| L1  | Single thread, 20M terms, 5GB |
+| L2  | Single-node perf, default user settings without MPI `1xcores`-shape. 1B terms, ~<100GB|
+| L3  | Multi-node, same problem as L2|
+| L4  | Strong/weak scaling. If L3 looks okay, don't bother running |
 
 ## Running one row
 
@@ -31,7 +29,7 @@ each hold their own operator, and selecting all four holds two per rank.
 
 ## Declaring the shape
 
-Three numbers describe every run: nodes `N`, ranks per node `R`, partitions per rank `P`, one core
+Nodes `N`, ranks per node `R`, partitions per rank `P`, one core
 per partition — so `R × P` is the cores per node and `P_total` = `N × R × P` is what a
 communication cost is a function of. **`N` and `R` come from the launcher; `P` you must export.**
 

From 8d79ff081acbad77dba7482cc7ee22bf40e17fc7 Mon Sep 17 00:00:00 2001
From: Aaron Miller <aaron.miller@algorithmiq.fi>
Date: Tue, 1 Sep 2026 09:40:50 +0100
Subject: [PATCH 06/14] =?UTF-8?q?perf(benches):=20=E2=9A=A1=20build=20the?=
 =?UTF-8?q?=20random=20graph=20once=20for=20build=5Fgraph,=20energy=20and?=
 =?UTF-8?q?=20gradient?=
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Selecting the three operations in one process built the operator twice: the
timed build_graph made its own, then the session built_graph fixture made a
second. build_graph now publishes the graph it timed, and energy and gradient
evaluate that one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---
 benches/bench_random.py |  6 +++++-
 benches/conftest.py     | 26 +++++++++++++++++++++++++-
 2 files changed, 30 insertions(+), 2 deletions(-)

diff --git a/benches/bench_random.py b/benches/bench_random.py
index 6a69843f..bdb4e4fd 100644
--- a/benches/bench_random.py
+++ b/benches/bench_random.py
@@ -26,6 +26,7 @@ def test_random_build_graph(
     bench_rounds,
     op_memory,
     record_opsize,
+    publish_graph,
 ):
     """Benchmark building the propagation graph from a fresh propagator."""
     last = []
@@ -47,8 +48,11 @@ def build(built):
         iterations=1,
     )
     op_memory.close(last[0])
-    assert record_opsize(last[0]) > 0
+    terms = record_opsize(last[0])
+    assert terms > 0
     assert last[0].graph_layers > 0
+    # Hands the timed graph to energy/gradient, so one process builds once.
+    publish_graph(last[0], terms)
 
 
 def test_random_propagate(
diff --git a/benches/conftest.py b/benches/conftest.py
index 0ebb954c..7a56ec75 100644
--- a/benches/conftest.py
+++ b/benches/conftest.py
@@ -136,6 +136,10 @@ def _spread(comm: Any, value: int) -> dict[str, int]:
     ("pare-threshold", float, None, "Edge-retention cutoff for the graph functionals."),
 )
 
+# Picture -> built graph, so one process can time build_graph, then energy, then gradient
+# against the graph it just built rather than building a second one.
+_GRAPH_CACHE: dict[str, Any] = {}
+
 _RESULTS: dict[str, Any] = {
     "meta": {},  # run configuration (ranks, threads, host, ...)
     "params": {},  # resolved random-problem hyperparameters
@@ -547,8 +551,12 @@ def built_graph(
 
     Session-scoped per picture so the graph is built once and shared across the
     read-only graph benchmarks (``energy``, ``gradient``), and records the operator
-    size for this picture while the graph is resident.
+    size for this picture while the graph is resident. A graph the timed
+    ``build_graph`` benchmark already published is reused instead of built again.
     """
+    if picture in _GRAPH_CACHE:
+        return _GRAPH_CACHE[picture]
+
     mp, circuit = build_random_propagator(
         random_problem, comm=bench_comm, schrodinger=picture == "schrodinger"
     )
@@ -557,9 +565,25 @@ def built_graph(
     # Under MPI the operator is partitioned, so sum the partitions.
     _record("opsize", picture, {"terms": _reduce_sum(bench_comm, mp.size())})
 
+    _GRAPH_CACHE[picture] = mp
     return mp
 
 
+@pytest.fixture(scope="session")
+def publish_graph(picture: str) -> Callable[[Any, int], None]:
+    """Return ``publish(propagator, terms)``, handing a timed build to ``built_graph``.
+
+    Takes the term count rather than recomputing it: ``record_opsize`` has already
+    reduced it, and a second collective reduce over a 1B-term operator is not free.
+    """
+
+    def _publish(mp: Any, terms: int) -> None:
+        _GRAPH_CACHE[picture] = mp
+        _record("opsize", picture, {"terms": terms})
+
+    return _publish
+
+
 @pytest.fixture(scope="session")
 def model_graph(
     model_configs: dict[str, Any], bench_comm: Any

From c349b299ea94d4f97367d6fee743d18c80b624ab Mon Sep 17 00:00:00 2001
From: Aaron Miller <aaron.miller@algorithmiq.fi>
Date: Tue, 1 Sep 2026 09:40:52 +0100
Subject: [PATCH 07/14] =?UTF-8?q?fix(bench-tools):=20=F0=9F=90=9B=20reject?=
 =?UTF-8?q?=20a=20--gen-length=20whose=20monomials=20are=20not=20Hermitian?=
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

A product of k Majoranas is Hermitian with real coefficients only for
k % 4 in (0, 1), so k=2, 3, 6, 7 reached the extension and surfaced as
"Non-Hermitian coeffs detected" from the propagator's constructor, naming
neither the option nor the rule. Verified over k=2..11.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---
 .../src/monoprop_bench_tools/models.py        | 17 +++++++++++++++-
 .../monoprop-bench-tools/tests/test_models.py | 20 +++++++++++++++++++
 2 files changed, 36 insertions(+), 1 deletion(-)

diff --git a/packages/monoprop-bench-tools/src/monoprop_bench_tools/models.py b/packages/monoprop-bench-tools/src/monoprop_bench_tools/models.py
index bd4bd440..0a5832fb 100644
--- a/packages/monoprop-bench-tools/src/monoprop_bench_tools/models.py
+++ b/packages/monoprop-bench-tools/src/monoprop_bench_tools/models.py
@@ -121,6 +121,18 @@ def parameters(self) -> np.ndarray:
         return np.asarray(self.circuit.parameters, dtype=float)
 
 
+def _check_gen_length(gen_length: int, option: str) -> None:
+    """Reject a generator length whose real-coefficient monomials are not Hermitian."""
+    if gen_length % 4 not in (0, 1):
+        msg = (
+            f"{option} gives length {gen_length}. A product of k Majoranas obeys "
+            f"(g_1...g_k)^dag = (-1)^(k(k-1)/2) g_1...g_k, so it is Hermitian only for "
+            f"k % 4 in (0, 1); at k={gen_length} the real coefficients drawn here are "
+            "anti-Hermitian and the propagator's constructor rejects them."
+        )
+        raise ValueError(msg)
+
+
 def _check_modes(num_modes: int, option: str) -> None:
     """Reject a size above the extension's compile-time mode limit."""
     if num_modes > MAX_NUM_MODES:
@@ -177,7 +189,9 @@ def make_random_problem(
     """Build a random observable and generator circuit for benchmarking.
 
     Args:
-        gen_length: Number of Majorana operators in each random generator.
+        gen_length: Number of Majorana operators in each random generator. Must satisfy
+            ``gen_length % 4 in (0, 1)``, the lengths at which a real-coefficient
+            Majorana product is Hermitian.
         obs_terms: Number of terms in the random observable.
         num_generators: Number of random generators in the circuit.
         num_modes: Number of fermionic modes (Majorana indices = ``2 * num_modes``).
@@ -189,6 +203,7 @@ def make_random_problem(
         A :class:`RandomProblem` bundling the observable, circuit, and cutoff.
     """
     _check_modes(num_modes, "--num-modes")
+    _check_gen_length(gen_length, "--gen-length")
     rng = np.random.default_rng(seed)
     num_majorana_indices = 2 * num_modes
 
diff --git a/packages/monoprop-bench-tools/tests/test_models.py b/packages/monoprop-bench-tools/tests/test_models.py
index 9c54865e..b0266d46 100644
--- a/packages/monoprop-bench-tools/tests/test_models.py
+++ b/packages/monoprop-bench-tools/tests/test_models.py
@@ -18,6 +18,8 @@
 
 import inspect
 
+import pytest
+
 from monoprop_bench_tools.models import build_random_propagator, make_random_problem
 
 
@@ -47,3 +49,21 @@ def test_built_graph_is_populated(serial_comm) -> None:
     propagator.build_graph(circuit)
     _n_cos_indices, n_cycles = propagator.graph_size()
     assert n_cycles > 0
+
+
+@pytest.mark.parametrize("gen_length", [4, 5, 8, 9])
+def test_hermitian_gen_lengths_are_accepted(gen_length: int) -> None:
+    problem = make_random_problem(
+        gen_length=gen_length, obs_terms=3, num_generators=2, num_modes=8, cutoff=6, seed=0
+    )
+    assert problem.circuit.system_size == 8
+
+
+@pytest.mark.parametrize("gen_length", [2, 3, 6, 7])
+def test_anti_hermitian_gen_lengths_raise_naming_the_option(gen_length: int) -> None:
+    # Unvalidated, the length reaches the extension and surfaces as a RuntimeError from the
+    # propagator's constructor, naming neither --gen-length nor the parity rule.
+    with pytest.raises(ValueError, match="--gen-length"):
+        make_random_problem(
+            gen_length=gen_length, obs_terms=3, num_generators=2, num_modes=8, cutoff=6, seed=0
+        )

From b9a45872cb6effea10b946b2ee97223599a3964b Mon Sep 17 00:00:00 2001
From: Aaron Miller <aaron.miller@algorithmiq.fi>
Date: Tue, 1 Sep 2026 09:55:11 +0100
Subject: [PATCH 08/14] =?UTF-8?q?docs(benches):=20=F0=9F=93=9D=20size=20th?=
 =?UTF-8?q?e=20Schr=C3=B6dinger=20rows=20to=20~100M=20terms=20and=20split?=
 =?UTF-8?q?=20energy=20from=20gradient?=
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The Schrödinger rows sat at 18.8M terms because --obs-terms, which sizes the
Heisenberg rows, is flat in a picture that evolves the state. Cutoff and mode
count are cliffs -- 7 and 180 both fail to fit a 242 GiB node where 6 and 160
cost 3.6 and 5.6 GiB -- so the rung is set by the gate count: 4000 generators
gives 97,631,757 terms in 16.2 GiB.

L2 and L3 now carry energy and gradient as separate rows; gradient is 3.4x
energy in the Heisenberg picture and 4.9x in the Schrödinger one, which one
combined cell hid.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---
 benches/LADDER.md                             | 113 ++++++++++++++----
 .../monoprop-bench-tools/tests/test_models.py |  15 ++-
 2 files changed, 105 insertions(+), 23 deletions(-)

diff --git a/benches/LADDER.md b/benches/LADDER.md
index e64228ff..46dc23d8 100644
--- a/benches/LADDER.md
+++ b/benches/LADDER.md
@@ -5,7 +5,7 @@
 | Group | What it measures |
 | --- | --- |
 | L1  | Single thread, 20M terms, 5GB |
-| L2  | Single-node perf, default user settings without MPI `1xcores`-shape. 1B terms, ~<100GB|
+| L2  | Single-node perf, default user settings without MPI `1xcores`-shape. 1B terms Heisenberg, ~100M Schrödinger, ≤110GB|
 | L3  | Multi-node, same problem as L2|
 | L4  | Strong/weak scaling. If L3 looks okay, don't bother running |
 
@@ -24,8 +24,10 @@ report. Reps are repeated invocations under different labels; leave `--bench-rou
 `pedantic` builds round *k+1*'s arguments before releasing round *k*'s and a second round holds two
 propagators.
 
-Run one group of operations per process at scale: `build_graph`/`propagate` and `energy`/`gradient`
-each hold their own operator, and selecting all four holds two per rank.
+`build_graph`, `energy` and `gradient` may be selected together in one process: the timed
+`build_graph` publishes the graph it built and the two functionals evaluate that one, so the
+process builds a single operator. Keep `propagate` out of that selection — it holds its own
+operator, so adding it to the other three holds two per rank.
 
 ## Declaring the shape
 
@@ -58,9 +60,55 @@ field of their config dataclass (`just bench --help` lists them); the random pro
 | `--hubbard-num-sites` | 2 × sites ≤ `MAX_NUM_MODES` | saturated from 60 at the default `lower_atol`; size Hubbard with `--hubbard-lower-atol` |
 | `--hubbard-observable-site` | `< --hubbard-num-sites` | a lattice position, not a constant: the default 46 sits 46/60 along the lattice, and shrinking the lattice without moving it changes the light cone. Out of range now raises |
 | `--pauli-num-qubits` | 127 only | `HEAVY_HEX_TOPOLOGY` is the fixed IBM Eagle coupling map, so any other count either indexes past the operator or leaves qubits uncoupled. Raises. Size this model with `--pauli-lower-atol` |
-| `--obs-terms` | any | an upper bound on the observable's terms, not an exact count: monomials are drawn independently and duplicates collapse, by about `obs_terms / 2·C(2·num_modes, gen_length)` — 0.06% at L1's 295k, 2.8% at L2's 14.75M. Deterministic for a fixed `--seed` |
+| `--obs-terms` | any | an upper bound on the observable's terms, not an exact count: monomials are drawn independently and duplicates collapse, by about `obs_terms / 2·C(2·num_modes, gen_length)` — 0.06% at L1's 295k, 2.8% at L2's 14.75M. Deterministic for a fixed `--seed`. **Heisenberg only** — see *Sizing the Schrödinger rows* |
+| `--num-generators` | any | the circuit's gate count, and the only usable size axis in the Schrödinger picture. It is also the length of the parameter vector, so `gradient` at 4000 generators does four times the per-term work it does at 1000 |
+| `--gen-length` | `k % 4` in {0, 1} | a product of `k` Majoranas satisfies `(g_1...g_k)^dag = (-1)^(k(k-1)/2) g_1...g_k`, so it is Hermitian with the real coefficients drawn here only at those lengths. Verified over k=2..11: 4, 5, 8, 9 build and 2, 3, 6, 7, 10, 11 raise. Out of range now raises naming the option, rather than surfacing as `Non-Hermitian coeffs detected` from inside the extension |
 | `--pare-threshold` | any, default off | edge-retention cutoff for the graph functionals (`energy`, `gradient`). Unset keeps the exact graph |
 
+### Sizing the Schrödinger rows
+
+The Schrödinger picture evolves the state, so the observable does not drive the operator:
+`--obs-terms` is flat there (18.85M / 18.75M / 18.83M across a 74× sweep) and sizes the Heisenberg
+rows only. Of the remaining axes, two are cliffs rather than dials — one step of either goes from a
+few GiB to not fitting a 242 GiB node, because what grows is a transient inside `build_graph` and
+not the retained operator:
+
+| axis | last value that fits | next value tried |
+| --- | --- | --- |
+| `--cutoff` | 6 — 18,853,861 terms, 3.6 GiB | 7 — SIGKILL |
+| `--num-modes` | 160 — 29,082,430 terms, 5.6 GiB | 180 — SIGKILL |
+
+`--num-generators` is the axis that works. At `--cutoff=6 --num-modes=142 --obs-terms=200000`,
+one node at 1×128, `build_graph`:
+
+| `--num-generators` | terms | ~s | ~GiB |
+| ---: | ---: | ---: | ---: |
+| 1000 | 18,853,861 | 1.3 | 3.6 |
+| 2000 | 24,707,186 | 3.2 | 4.2 |
+| 3000 | 42,566,127 | 6.1 | 7.2 |
+| 4000 | 97,631,757 | 13.3 | 16.2 |
+| 5000 | 261,929,069 | 30.6 | 32.2 |
+| 6000 | 737,159,992 | 79.9 | 82.7 |
+| **6200** | **907,357,991** | **98.1** | **102.2** |
+| 6400 | 1,115,757,089 | 117.0 | 138.5 |
+| 8000 | SIGKILL | | > 242 |
+
+L2 and L3 take the 6200-generator row: it is the largest that fits the ~110 GiB one-node budget
+the other L2 rows are held to, and 6400 is what going over looks like — 1.12B terms, but 138.5 GiB.
+So the Schrödinger rows sit at 0.91B against the Heisenberg rows' 0.95B, matched in size rather
+than in gate count.
+
+The axis accelerates — ×1.31, ×1.72, ×2.29, ×2.68 per extra 1000 gates — so read a size off this
+table rather than interpolating, and do not size the rung from two points: 1000→2000 alone
+predicts ~73,000 generators for 100M terms, where 4000 already reaches it. The counts are
+deterministic at a fixed `--seed`; 4000 reproduced to the term across two jobs.
+
+**A gate count is not a free parameter here.** `--num-generators` is also the length of the
+parameter vector, so the 6200-generator Schrödinger rows return a 6200-long gradient where the
+1000-generator Heisenberg rows return a 1000-long one. The two pictures' `gradient` cells are
+matched in terms and in memory, not in the work the functional does; do not read one against the
+other as a per-term ratio.
+
 ## L1 — one thread
 
 One node, one rank, one partition: `export monoprop_PARTITIONS=1 monoprop_NUM_THREADS=1`.
@@ -89,10 +137,24 @@ they are where the operations separate.
 | hubbard `propagate` | `--hubbard-cutoff=10 --hubbard-lower-atol=3.38e-06` | `test_model_propagate and hubbard` | 1,001,661,534 | 140 | 60 |
 | pauli `propagate` | `--pauli-cutoff=14 --pauli-lower-atol=8.9e-06` | `test_model_propagate and pauli` | 985,970,588 | 90 | 80 |
 | random `propagate` | `--num-generators=1000 --num-modes=142 --cutoff=6 --obs-terms=14750000` | `test_random_propagate and heisenberg` | 948,937,993 | 50 | 95 |
-| random `build_graph` | *(same)* | `test_random_build_graph and heisenberg` | 948,937,993 | 50 | 110 |
-| random `energy`+`gradient` | *(same)* | `(test_random_energy or test_random_gradient) and heisenberg` | 948,937,993 | 40 | 110 |
-| Schrödinger `build_graph` | *(same)* | `test_random_build_graph and schrodinger` | 18,825,440 | 1.5 | 17 |
-| Schrödinger `energy`+`gradient` | *(same)* | `(test_random_energy or test_random_gradient) and schrodinger` | 18,825,440 | 0.3 | 11 |
+| random `build_graph` | *(same)* | `test_random_build_graph and heisenberg` | 948,937,993 | 50 | 109 |
+| random `energy` | *(same)* | `test_random_energy and heisenberg` | 948,937,993 | 9.9 | 102 |
+| random `gradient` | *(same)* | `test_random_gradient and heisenberg` | 948,937,993 | 34 | 110 |
+| Schrödinger `build_graph` | `--num-generators=4000 --num-modes=142 --cutoff=6 --obs-terms=200000` | `test_random_build_graph and schrodinger` | 97,631,757 | 13 | 16 |
+| Schrödinger `energy` | *(same)* | `test_random_energy and schrodinger` | 97,631,757 | 0.8 | 15 |
+| Schrödinger `gradient` | *(same)* | `test_random_gradient and schrodinger` | 97,631,757 | 4.0 | 19 |
+
+Medians of three reps, arm order flipped between reps. `energy` and `gradient` are separate rows
+because they are not one cost: `gradient` is 3.4x `energy` in the Heisenberg picture and 4.9x in
+the Schrödinger one.
+
+**The Schrödinger rows carry their own flags**, differing in the gate count. `--obs-terms` cannot
+size that picture, so the rung is set by `--num-generators=4000` (see *Sizing the Schrödinger
+rows*); the smaller observable is what drops its memory, not the smaller operator. Two consequences
+worth reading before comparing the pictures: the gradient's parameter vector is one entry per
+generator, so the Schrödinger `gradient` differentiates 4000 parameters against the Heisenberg
+row's 1000 — unequal work, not just unequal size — and at 97.6M against 949M terms the two pictures
+are an order of magnitude apart by construction.
 
 ## L3 — several nodes, the same problems
 
@@ -104,20 +166,31 @@ Same flags and same `-k` as L2, so only the measured cells differ. Term counts a
 geometry-independent (`cpp/tests/partition_equivalence_tests.cpp`), which is why one calibration
 serves every shape and a term-count disagreement means the *problem* changed, not the topology.
 
-| row | terms | ~s | ~GiB/node |
-| --- | ---: | ---: | ---: |
-| hubbard `propagate` | 1,001,661,534 | 35 | 18 |
-| pauli `propagate` | 985,970,588 | 23 | 25 |
-| random `propagate` | 948,937,993 | 13 | 100 |
-| random `build_graph` | 948,937,993 | 13 | 100 |
-| random `energy`+`gradient` | 948,937,993 | 9 | 60 |
-| Schrödinger `build_graph` | 18,825,440 | 1.1 | 80 |
-| Schrödinger `energy`+`gradient` | 18,825,440 | 0.15 | 40 |
+| row | terms | ~s | ~GiB/node | ~GiB worst rank |
+| --- | ---: | ---: | ---: | ---: |
+| hubbard `propagate` | 1,001,661,534 | 35 | 18 | |
+| pauli `propagate` | 985,970,588 | 23 | 25 | |
+| random `propagate` | 948,937,993 | 13 | 100 | |
+| random `build_graph` | 948,937,993 | 14 | 105 | 13.1 |
+| random `energy` | 948,937,993 | 2.3 | 61 | 7.7 |
+| random `gradient` | 948,937,993 | 7.6 | 63 | 7.9 |
+| Schrödinger `build_graph` | 97,631,757 | 4.7 | 7.5 | 0.9 |
+| Schrödinger `energy` | 97,631,757 | 0.35 | 6.7 | 0.8 |
+| Schrödinger `gradient` | 97,631,757 | 1.2 | 7.9 | 1.0 |
 
 Measured at `N`=4, `R`=8, `P`=16 (medians of two reps) — a ballpark for sizing a job at that
-shape, not a target. Memory per node does not fall the way time does: there is a per-process
-resident floor of roughly 9.5 GiB, paid once per rank whatever the problem, so a row whose
-operation is cheap in memory pays more per node at 8 ranks than at 1.
+shape, not a target. Term counts are identical to L2's, which is the geometry-independence check.
+
+Memory per node does not fall the way time does, but there is **no fixed per-rank floor**: the
+Heisenberg rows pay 13.1 GiB on their worst rank and the Schrödinger rows 0.9 GiB, on the same
+shape. An earlier revision of this table quoted a ~9.5 GiB per-rank floor "paid whatever the
+problem", which the Schrödinger rows here refute by an order of magnitude. Size a job from the row
+you are running.
+
+The Schrödinger rows are also where the flag split pays off: 7.5 GiB/node here against the
+80 GiB/node an earlier revision recorded at 18.8M terms, at 5.2x the terms. That row inherited the
+Heisenberg arm's 14.75M-term observable and every rank on the node held a copy of it, while it
+contributed nothing to the evolved state.
 
 ## L4 — strong and weak scaling
 
diff --git a/packages/monoprop-bench-tools/tests/test_models.py b/packages/monoprop-bench-tools/tests/test_models.py
index b0266d46..02a6c6be 100644
--- a/packages/monoprop-bench-tools/tests/test_models.py
+++ b/packages/monoprop-bench-tools/tests/test_models.py
@@ -19,7 +19,6 @@
 import inspect
 
 import pytest
-
 from monoprop_bench_tools.models import build_random_propagator, make_random_problem
 
 
@@ -54,7 +53,12 @@ def test_built_graph_is_populated(serial_comm) -> None:
 @pytest.mark.parametrize("gen_length", [4, 5, 8, 9])
 def test_hermitian_gen_lengths_are_accepted(gen_length: int) -> None:
     problem = make_random_problem(
-        gen_length=gen_length, obs_terms=3, num_generators=2, num_modes=8, cutoff=6, seed=0
+        gen_length=gen_length,
+        obs_terms=3,
+        num_generators=2,
+        num_modes=8,
+        cutoff=6,
+        seed=0,
     )
     assert problem.circuit.system_size == 8
 
@@ -65,5 +69,10 @@ def test_anti_hermitian_gen_lengths_raise_naming_the_option(gen_length: int) ->
     # propagator's constructor, naming neither --gen-length nor the parity rule.
     with pytest.raises(ValueError, match="--gen-length"):
         make_random_problem(
-            gen_length=gen_length, obs_terms=3, num_generators=2, num_modes=8, cutoff=6, seed=0
+            gen_length=gen_length,
+            obs_terms=3,
+            num_generators=2,
+            num_modes=8,
+            cutoff=6,
+            seed=0,
         )

From 05eec0d33ff605343e02ae4dd59f593a3ba71bde Mon Sep 17 00:00:00 2001
From: Aaron Miller <aaron.miller@algorithmiq.fi>
Date: Tue, 1 Sep 2026 10:30:12 +0100
Subject: [PATCH 09/14] =?UTF-8?q?docs(benches):=20=F0=9F=93=9D=20size=20th?=
 =?UTF-8?q?e=20Schr=C3=B6dinger=20rows=20against=20gradient,=20not=20build?=
 =?UTF-8?q?=5Fgraph?=
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The rung was set from a build_graph sweep, which is the wrong operation to
size it by: at 6200 generators build_graph peaks at 102.7 GiB and gradient in
the same process peaks at 143.1 GiB. Re-measured whole-row peaks and took
5800 generators (597,445,055 terms, 99.7 GiB), the largest that fits the
~110 GiB budget the Heisenberg rows sit at.

Records both cliffs found on the way (cutoff 7 and 180 modes each SIGKILL a
242 GiB node where their predecessors cost 3.6 and 5.6 GiB), the generator
axis from 1000 to 8000, and why the last operation in a shared process reads
highest: it starts at its predecessor's high-water mark, so only 28.8 GiB of
gradient's 143.1 is its own.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---
 benches/LADDER.md | 105 +++++++++++++++++++++++++++++-----------------
 1 file changed, 66 insertions(+), 39 deletions(-)

diff --git a/benches/LADDER.md b/benches/LADDER.md
index 46dc23d8..d4b7ae53 100644
--- a/benches/LADDER.md
+++ b/benches/LADDER.md
@@ -5,7 +5,7 @@
 | Group | What it measures |
 | --- | --- |
 | L1  | Single thread, 20M terms, 5GB |
-| L2  | Single-node perf, default user settings without MPI `1xcores`-shape. 1B terms Heisenberg, ~100M Schrödinger, ≤110GB|
+| L2  | Single-node perf, default user settings without MPI `1xcores`-shape. 0.95B terms Heisenberg, 0.60B Schrödinger, both ≤110GB|
 | L3  | Multi-node, same problem as L2|
 | L4  | Strong/weak scaling. If L3 looks okay, don't bother running |
 
@@ -61,7 +61,7 @@ field of their config dataclass (`just bench --help` lists them); the random pro
 | `--hubbard-observable-site` | `< --hubbard-num-sites` | a lattice position, not a constant: the default 46 sits 46/60 along the lattice, and shrinking the lattice without moving it changes the light cone. Out of range now raises |
 | `--pauli-num-qubits` | 127 only | `HEAVY_HEX_TOPOLOGY` is the fixed IBM Eagle coupling map, so any other count either indexes past the operator or leaves qubits uncoupled. Raises. Size this model with `--pauli-lower-atol` |
 | `--obs-terms` | any | an upper bound on the observable's terms, not an exact count: monomials are drawn independently and duplicates collapse, by about `obs_terms / 2·C(2·num_modes, gen_length)` — 0.06% at L1's 295k, 2.8% at L2's 14.75M. Deterministic for a fixed `--seed`. **Heisenberg only** — see *Sizing the Schrödinger rows* |
-| `--num-generators` | any | the circuit's gate count, and the only usable size axis in the Schrödinger picture. It is also the length of the parameter vector, so `gradient` at 4000 generators does four times the per-term work it does at 1000 |
+| `--num-generators` | any | the circuit's gate count, and the only usable size axis in the Schrödinger picture. It is also the length of the parameter vector, so `gradient` at 5800 generators does nearly six times the per-term work it does at 1000 |
 | `--gen-length` | `k % 4` in {0, 1} | a product of `k` Majoranas satisfies `(g_1...g_k)^dag = (-1)^(k(k-1)/2) g_1...g_k`, so it is Hermitian with the real coefficients drawn here only at those lengths. Verified over k=2..11: 4, 5, 8, 9 build and 2, 3, 6, 7, 10, 11 raise. Out of range now raises naming the option, rather than surfacing as `Non-Hermitian coeffs detected` from inside the extension |
 | `--pare-threshold` | any, default off | edge-retention cutoff for the graph functionals (`energy`, `gradient`). Unset keeps the exact graph |
 
@@ -79,24 +79,39 @@ not the retained operator:
 | `--num-modes` | 160 — 29,082,430 terms, 5.6 GiB | 180 — SIGKILL |
 
 `--num-generators` is the axis that works. At `--cutoff=6 --num-modes=142 --obs-terms=200000`,
-one node at 1×128, `build_graph`:
-
-| `--num-generators` | terms | ~s | ~GiB |
-| ---: | ---: | ---: | ---: |
-| 1000 | 18,853,861 | 1.3 | 3.6 |
-| 2000 | 24,707,186 | 3.2 | 4.2 |
-| 3000 | 42,566,127 | 6.1 | 7.2 |
-| 4000 | 97,631,757 | 13.3 | 16.2 |
-| 5000 | 261,929,069 | 30.6 | 32.2 |
-| 6000 | 737,159,992 | 79.9 | 82.7 |
-| **6200** | **907,357,991** | **98.1** | **102.2** |
-| 6400 | 1,115,757,089 | 117.0 | 138.5 |
-| 8000 | SIGKILL | | > 242 |
-
-L2 and L3 take the 6200-generator row: it is the largest that fits the ~110 GiB one-node budget
-the other L2 rows are held to, and 6400 is what going over looks like — 1.12B terms, but 138.5 GiB.
-So the Schrödinger rows sit at 0.91B against the Heisenberg rows' 0.95B, matched in size rather
-than in gate count.
+one node at 1×128. Term counts are geometry- and operation-independent; the time and memory
+columns are not, so each row says what was measured — `build` is `build_graph` alone, `row` is all
+three operations in one process, which is what has to fit:
+
+| `--num-generators` | terms | ~s | ~GiB | measured |
+| ---: | ---: | ---: | ---: | --- |
+| 1000 | 18,853,861 | 1.3 | 3.6 | build |
+| 2000 | 24,707,186 | 3.2 | 4.2 | build |
+| 3000 | 42,566,127 | 6.1 | 7.2 | build |
+| 4000 | 97,631,757 | 13.3 | 16.2 | build |
+| 5000 | 261,929,069 | 30.6 | 32.2 | build |
+| 5600 | 485,959,982 | 97.9 | 78.0 | row |
+| **5800** | **597,445,055** | **115.9** | **99.7** | **row** |
+| 6000 | 737,159,992 | 142.8 | 115.1 | row |
+| 6200 | 907,357,991 | 172.1 | 143.1 | row |
+| 6400 | 1,115,757,089 | 117.0 | 138.5 | build |
+| 8000 | — | — | > 242 (SIGKILL) | build |
+
+L2 and L3 take the 5800-generator row: it is the largest whose whole-row peak fits the ~110 GiB
+one-node budget the other L2 rows are held to. 6000 (115.1 GiB) and 6200 (143.1 GiB) clear 1B terms
+at 6400 but not the budget.
+
+**Size this rung against `gradient`, not `build_graph`.** At 6200 generators `build_graph` peaks at
+102.7 GiB and looks like it fits, while `gradient` in the same process peaks at 143.1 GiB. Two
+things make the last operation the expensive one, and only the second is about the gradient:
+
+- The three operations share a process, and peak RSS follows capacity rather than live bytes, so an
+  operation starts at the high-water mark its predecessors left. Measured at 6200: `gradient`'s
+  floor was 114.31 GiB against `energy`'s peak of 114.32 — the same bytes, not new ones. Of that
+  143.1 GiB row peak only 28.8 GiB is `gradient`'s own transient (`opmemdelta`).
+- That transient still tracks the *circuit*, not the operator: 28.8 GiB at 6200 generators against
+  7.9 GiB at 1000, on an operator that is slightly smaller (907M terms against 949M). Sizing this
+  picture by gate count therefore buys terms and gradient memory together.
 
 The axis accelerates — ×1.31, ×1.72, ×2.29, ×2.68 per extra 1000 gates — so read a size off this
 table rather than interpolating, and do not size the rung from two points: 1000→2000 alone
@@ -104,7 +119,7 @@ predicts ~73,000 generators for 100M terms, where 4000 already reaches it. The c
 deterministic at a fixed `--seed`; 4000 reproduced to the term across two jobs.
 
 **A gate count is not a free parameter here.** `--num-generators` is also the length of the
-parameter vector, so the 6200-generator Schrödinger rows return a 6200-long gradient where the
+parameter vector, so the 5800-generator Schrödinger rows return a 5800-long gradient where the
 1000-generator Heisenberg rows return a 1000-long one. The two pictures' `gradient` cells are
 matched in terms and in memory, not in the work the functional does; do not read one against the
 other as a per-term ratio.
@@ -140,21 +155,29 @@ they are where the operations separate.
 | random `build_graph` | *(same)* | `test_random_build_graph and heisenberg` | 948,937,993 | 50 | 109 |
 | random `energy` | *(same)* | `test_random_energy and heisenberg` | 948,937,993 | 9.9 | 102 |
 | random `gradient` | *(same)* | `test_random_gradient and heisenberg` | 948,937,993 | 34 | 110 |
-| Schrödinger `build_graph` | `--num-generators=4000 --num-modes=142 --cutoff=6 --obs-terms=200000` | `test_random_build_graph and schrodinger` | 97,631,757 | 13 | 16 |
-| Schrödinger `energy` | *(same)* | `test_random_energy and schrodinger` | 97,631,757 | 0.8 | 15 |
-| Schrödinger `gradient` | *(same)* | `test_random_gradient and schrodinger` | 97,631,757 | 4.0 | 19 |
+| Schrödinger `build_graph` | `--num-generators=5800 --num-modes=142 --cutoff=6 --obs-terms=200000` | `test_random_build_graph and schrodinger` | 597,445,055 | 66 | 73 |
+| Schrödinger `energy` | *(same)* | `test_random_energy and schrodinger` | 597,445,055 | 11 | 79 |
+| Schrödinger `gradient` | *(same)* | `test_random_gradient and schrodinger` | 597,445,055 | 39 | 100 |
 
 Medians of three reps, arm order flipped between reps. `energy` and `gradient` are separate rows
-because they are not one cost: `gradient` is 3.4x `energy` in the Heisenberg picture and 4.9x in
+because they are not one cost: `gradient` is 3.4x `energy` in the Heisenberg picture and 3.4x in
 the Schrödinger one.
 
-**The Schrödinger rows carry their own flags**, differing in the gate count. `--obs-terms` cannot
-size that picture, so the rung is set by `--num-generators=4000` (see *Sizing the Schrödinger
-rows*); the smaller observable is what drops its memory, not the smaller operator. Two consequences
-worth reading before comparing the pictures: the gradient's parameter vector is one entry per
-generator, so the Schrödinger `gradient` differentiates 4000 parameters against the Heisenberg
-row's 1000 — unequal work, not just unequal size — and at 97.6M against 949M terms the two pictures
-are an order of magnitude apart by construction.
+`build_graph`, `energy` and `gradient` share one process and one build — the graph the timed
+`build_graph` produces is what the other two evaluate — so run them with one selector, not three.
+`propagate` holds its own operator and stays out of that group.
+
+**The Schrödinger rows carry their own flags**, differing only in the gate count. `--obs-terms`
+cannot size that picture, so the rung is set by `--num-generators=5800` (see *Sizing the Schrödinger
+rows*), which is the largest gate count whose whole-row peak stays inside the ~110 GiB budget the
+Heisenberg rows sit at. Two consequences worth reading before comparing the pictures:
+
+- The gradient's parameter vector is one entry per generator, so the Schrödinger `gradient`
+  differentiates 5800 parameters against the Heisenberg row's 1000. Unequal work, not just unequal
+  size.
+- Memory per term is higher in the Schrödinger picture — 0.167 GiB/Mterm at the `gradient` peak
+  against the Heisenberg rows' 0.115 — which is why matching the two pictures on memory leaves them
+  at 597M against 949M terms rather than at equal counts.
 
 ## L3 — several nodes, the same problems
 
@@ -174,24 +197,28 @@ serves every shape and a term-count disagreement means the *problem* changed, no
 | random `build_graph` | 948,937,993 | 14 | 105 | 13.1 |
 | random `energy` | 948,937,993 | 2.3 | 61 | 7.7 |
 | random `gradient` | 948,937,993 | 7.6 | 63 | 7.9 |
-| Schrödinger `build_graph` | 97,631,757 | 4.7 | 7.5 | 0.9 |
-| Schrödinger `energy` | 97,631,757 | 0.35 | 6.7 | 0.8 |
-| Schrödinger `gradient` | 97,631,757 | 1.2 | 7.9 | 1.0 |
+| Schrödinger `build_graph` | 597,445,055 | 20 | 26 | 3.3 |
+| Schrödinger `energy` | 597,445,055 | 1.9 | 26 | 3.3 |
+| Schrödinger `gradient` | 597,445,055 | 8.9 | 36 | 4.5 |
 
 Measured at `N`=4, `R`=8, `P`=16 (medians of two reps) — a ballpark for sizing a job at that
 shape, not a target. Term counts are identical to L2's, which is the geometry-independence check.
 
 Memory per node does not fall the way time does, but there is **no fixed per-rank floor**: the
-Heisenberg rows pay 13.1 GiB on their worst rank and the Schrödinger rows 0.9 GiB, on the same
+Heisenberg rows pay 13.1 GiB on their worst rank and the Schrödinger rows 3.3 GiB, on the same
 shape. An earlier revision of this table quoted a ~9.5 GiB per-rank floor "paid whatever the
-problem", which the Schrödinger rows here refute by an order of magnitude. Size a job from the row
+problem", which the Schrödinger rows here refute by a factor of four. Size a job from the row
 you are running.
 
-The Schrödinger rows are also where the flag split pays off: 7.5 GiB/node here against the
-80 GiB/node an earlier revision recorded at 18.8M terms, at 5.2x the terms. That row inherited the
+The Schrödinger rows are also where the flag split pays off: 26 GiB/node here against the
+80 GiB/node an earlier revision recorded at 18.8M terms, at 32x the terms. That row inherited the
 Heisenberg arm's 14.75M-term observable and every rank on the node held a copy of it, while it
 contributed nothing to the evolved state.
 
+The single-node budget, not this shape, is what sizes the Schrödinger rung: at `N`=4 its worst rank
+holds 4.5 GiB and the row would fit many times over. L2 and L3 share one flag set so the term
+counts stay comparable, so L2 is the binding constraint.
+
 ## L4 — strong and weak scaling
 
 Hubbard `propagate` only: a communication cost is a function of `P_total`, so varying the model as

From d01d9e596a85dd0a00cd4d7732e4781e186c5bfe Mon Sep 17 00:00:00 2001
From: Aaron Miller <aaron.miller@algorithmiq.fi>
Date: Tue, 1 Sep 2026 12:47:09 +0100
Subject: [PATCH 10/14] =?UTF-8?q?docs(benches):=20=F0=9F=93=9D=20split=20L?=
 =?UTF-8?q?2=20into=20one=20rank=20and=20one=20node=20of=20ranks?=
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

L2 -> L3 moved two things at once: it introduced multiple processes and it
introduced the network, so anything that appeared at L3 was attributable to
neither. L2 splits into L2a (one rank, partitions = threads = cores, which is
the old L2 unchanged) and L2b (one node at R ranks x P partitions, R x P =
cores, which is L3's shape at N = 1). Each edge of the ladder now moves one
thing: partitions, processes, the network, then node count.

L2b is measured at N=1, R=8, P=16 on a 128-core node, two reps with the cell
order flipped. Term counts reproduce L2a's and L3's exactly, which is the
geometry-independence check the rung relies on.

8 x 16 is stated as a choice, not derived as an optimum: it is one rank per NUMA
domain on this machine, and it is L3's shape, so L2b -> L3 differs only in the
node count. The section says as much rather than implying a sweep.

Two findings worth the reader's attention, both recorded next to the table:

- The Heisenberg rows cost 3.6-8.1 GiB per rank beyond their share of the
  operator, the Schrodinger rows 0.2-0.6, on one shape and one binary. The flag
  sets differ in the observable, 14.75M terms against 200k. The raw observable
  is well under a GiB, so something indexed by it scales with it; that is not
  attributed further here.
- Consequently L2b, not L2a, is the memory-worst rung for the Heisenberg rows,
  because the per-rank cost is paid R times with no extra nodes to spread it
  over. An earlier draft of this change claimed a row fitting L2a fits L2b, and
  the measurement refutes it.

The GiB/node column is a sum over ranks, as L3's is; reading it against L2a's
single-process peak would be a sum-against-max comparison, so the table says so.

Also: a new trap, P above the cores a rank can see, which is what carrying L2a's
monoprop_PARTITIONS into an L2b launch does; and the multi-rank skeleton now
names the interpreter directly, because uv run re-resolves the environment in
every rank.

Assisted-by: ClaudeCode:claude-opus-5
---
 benches/LADDER.md                | 153 ++++++++++++++++++++++++++-----
 docs/content/docs/benchmarks.mdx |   6 +-
 2 files changed, 134 insertions(+), 25 deletions(-)

diff --git a/benches/LADDER.md b/benches/LADDER.md
index d4b7ae53..2889101b 100644
--- a/benches/LADDER.md
+++ b/benches/LADDER.md
@@ -5,8 +5,9 @@
 | Group | What it measures |
 | --- | --- |
 | L1  | Single thread, 20M terms, 5GB |
-| L2  | Single-node perf, default user settings without MPI `1xcores`-shape. 0.95B terms Heisenberg, 0.60B Schrödinger, both ≤110GB|
-| L3  | Multi-node, same problem as L2|
+| L2a | Single-node perf, default user settings without MPI `1xcores`-shape. 0.95B terms Heisenberg, 0.60B Schrödinger, both ≤110GB|
+| L2b | The L2a problems on one node at `R × P` — MPI with no network in it |
+| L3  | Multi-node, same problem as L2b|
 | L4  | Strong/weak scaling. If L3 looks okay, don't bother running |
 
 ## Running one row
@@ -43,6 +44,16 @@ Above one rank this is mandatory: `benches/conftest.py` refuses to start a multi
 `monoprop_PARTITIONS` unset, because the engine's own default is `ranks == 1 ? cores : 1` and an
 unset knob would silently measure one partition per rank at a plausible wall time.
 
+The rungs fix the shape as follows, so that each edge of the ladder moves one thing:
+
+| rung | `N` | `R` | `P` | the edge below it |
+| --- | ---: | ---: | ---: | --- |
+| L1 | 1 | 1 | 1 | — |
+| L2a | 1 | 1 | cores | in-process partitioning |
+| L2b | 1 | yours | cores / `R` | per process, no network |
+| L3 | yours | L2b's | L2b's | the network |
+| L4 | 1…64 | L2b's | L2b's | scaling in `N` |
+
 The shape is recorded with every run — `ranks`, `nodes`, `ranks_per_node`, `partitions_env` and
 `monoprop_threads` in `results/<label>.json`'s `meta`, surfaced as the Configuration table of
 `REPORT.md`. Check it before comparing two rows: a ladder whose points did not all run at the same
@@ -60,7 +71,7 @@ field of their config dataclass (`just bench --help` lists them); the random pro
 | `--hubbard-num-sites` | 2 × sites ≤ `MAX_NUM_MODES` | saturated from 60 at the default `lower_atol`; size Hubbard with `--hubbard-lower-atol` |
 | `--hubbard-observable-site` | `< --hubbard-num-sites` | a lattice position, not a constant: the default 46 sits 46/60 along the lattice, and shrinking the lattice without moving it changes the light cone. Out of range now raises |
 | `--pauli-num-qubits` | 127 only | `HEAVY_HEX_TOPOLOGY` is the fixed IBM Eagle coupling map, so any other count either indexes past the operator or leaves qubits uncoupled. Raises. Size this model with `--pauli-lower-atol` |
-| `--obs-terms` | any | an upper bound on the observable's terms, not an exact count: monomials are drawn independently and duplicates collapse, by about `obs_terms / 2·C(2·num_modes, gen_length)` — 0.06% at L1's 295k, 2.8% at L2's 14.75M. Deterministic for a fixed `--seed`. **Heisenberg only** — see *Sizing the Schrödinger rows* |
+| `--obs-terms` | any | an upper bound on the observable's terms, not an exact count: monomials are drawn independently and duplicates collapse, by about `obs_terms / 2·C(2·num_modes, gen_length)` — 0.06% at L1's 295k, 2.8% at L2a's 14.75M. Deterministic for a fixed `--seed`. **Heisenberg only** — see *Sizing the Schrödinger rows* |
 | `--num-generators` | any | the circuit's gate count, and the only usable size axis in the Schrödinger picture. It is also the length of the parameter vector, so `gradient` at 5800 generators does nearly six times the per-term work it does at 1000 |
 | `--gen-length` | `k % 4` in {0, 1} | a product of `k` Majoranas satisfies `(g_1...g_k)^dag = (-1)^(k(k-1)/2) g_1...g_k`, so it is Hermitian with the real coefficients drawn here only at those lengths. Verified over k=2..11: 4, 5, 8, 9 build and 2, 3, 6, 7, 10, 11 raise. Out of range now raises naming the option, rather than surfacing as `Non-Hermitian coeffs detected` from inside the extension |
 | `--pare-threshold` | any, default off | edge-retention cutoff for the graph functionals (`energy`, `gradient`). Unset keeps the exact graph |
@@ -97,8 +108,8 @@ three operations in one process, which is what has to fit:
 | 6400 | 1,115,757,089 | 117.0 | 138.5 | build |
 | 8000 | — | — | > 242 (SIGKILL) | build |
 
-L2 and L3 take the 5800-generator row: it is the largest whose whole-row peak fits the ~110 GiB
-one-node budget the other L2 rows are held to. 6000 (115.1 GiB) and 6200 (143.1 GiB) clear 1B terms
+L2a, L2b and L3 take the 5800-generator row: it is the largest whose whole-row peak fits the
+~110 GiB one-node budget the other L2a rows are held to. 6000 (115.1 GiB) and 6200 (143.1 GiB) clear 1B terms
 at 6400 but not the budget.
 
 **Size this rung against `gradient`, not `build_graph`.** At 6200 generators `build_graph` peaks at
@@ -141,11 +152,13 @@ either alone.
 | random `gradient` | `--num-generators=1000 --num-modes=142 --cutoff=6 --obs-terms=295000` | `test_random_gradient and heisenberg` | 19,902,244 | 11 | 2.5 |
 | random `gradient`, pared | *(same)* `--pare-threshold=1e-10` | `test_random_gradient and heisenberg` | 19,902,244 | 0.8 | 2.8 |
 
-## L2 — one node, ~1B terms
+## L2a — one node, one rank, ~1B terms
 
 One node, one rank, every core: `export monoprop_PARTITIONS=128 monoprop_NUM_THREADS=128`
-(substitute your core count). The random rows are the only ones carrying all four operations, so
-they are where the operations separate.
+(substitute your core count). No MPI, so `P` alone carries the parallelism — this is the shape a
+user who never launches a rank gets, and the one whose partition count the engine would have
+picked on its own. The random rows are the only ones carrying all four operations, so they are
+where the operations separate.
 
 | row | flags | `-k` | terms | ~s | ~GiB |
 | --- | --- | --- | ---: | ---: | ---: |
@@ -163,6 +176,8 @@ Medians of three reps, arm order flipped between reps. `energy` and `gradient` a
 because they are not one cost: `gradient` is 3.4x `energy` in the Heisenberg picture and 3.4x in
 the Schrödinger one.
 
+These nine rows are the problem set for L2b and L3 as well; only the shape changes.
+
 `build_graph`, `energy` and `gradient` share one process and one build — the graph the timed
 `build_graph` produces is what the other two evaluate — so run them with one selector, not three.
 `propagate` holds its own operator and stays out of that group.
@@ -170,7 +185,10 @@ the Schrödinger one.
 **The Schrödinger rows carry their own flags**, differing only in the gate count. `--obs-terms`
 cannot size that picture, so the rung is set by `--num-generators=5800` (see *Sizing the Schrödinger
 rows*), which is the largest gate count whose whole-row peak stays inside the ~110 GiB budget the
-Heisenberg rows sit at. Two consequences worth reading before comparing the pictures:
+Heisenberg rows sit at. That budget sizes the *flags*, which every rung then shares — but it does
+not bound the other rungs' memory, and measurement says L2b's is higher: per-rank overhead is paid
+`R` times and `N`=1 gives it no extra nodes to spread over. Size an L2b or L3 job from its own
+table. Two consequences worth reading before comparing the pictures:
 
 - The gradient's parameter vector is one entry per generator, so the Schrödinger `gradient`
   differentiates 5800 parameters against the Heisenberg row's 1000. Unequal work, not just unequal
@@ -179,15 +197,89 @@ Heisenberg rows sit at. Two consequences worth reading before comparing the pict
   against the Heisenberg rows' 0.115 — which is why matching the two pictures on memory leaves them
   at 597M against 949M terms rather than at equal counts.
 
+## L2b — one node, `R` ranks × `P` partitions
+
+The L2a problems again, on one node, with the cores split across processes instead of held by one:
+`R × P` = the core count and `N` = 1. L2a → L2b is therefore the *per-process* step with no network
+in it, and L2b → L3 is the network step. Before this rung existed the ladder went from one rank
+straight to several nodes and moved both at once, which left anything that appeared at L3
+attributable to neither.
+
+Same flags and same `-k` as L2a — only the shape differs. Term counts are geometry-independent
+(`cpp/tests/partition_equivalence_tests.cpp`), so a term count that disagrees with L2a's means the
+*problem* changed, not the topology.
+
+### Choosing `R` and `P`
+
+Two constraints, and then one choice:
+
+- **`R × P` = the cores per node.** This is what makes the rung a comparison rather than a separate
+  measurement: the core count is held, so the only thing separating it from L2a is how many
+  processes hold those cores. Any other product is a valid run, just not this rung.
+- **`P` ≤ the cores one rank can see**, which under `--cpu-bind=cores` is its `--cpus-per-task`.
+  Above that the engine places nothing at all, at 13.8x to 24.6x, and says so only on C++ stderr —
+  see *Traps*. Note that this is the constraint L2a's `P` violates if it is left in the
+  environment: `P` here is `cores / R`.
+- **One rank per NUMA domain**, which then fixes both numbers on a given machine. The node these
+  tables were measured on is 8 domains × 16 cores, so `R`=8 and `P`=16.
+
+The third is a choice rather than a measured optimum, made for two reasons. It is the layout the
+library ships in, and it is **L3's shape** — L3 runs the same `R` and `P` at `N` > 1, so the two
+rungs differ in the node count and nothing else. If you change the split, change it in both rungs
+or the comparison they exist for is gone.
+
+| row | terms | ~s | ~GiB/node | ~GiB worst rank |
+| --- | ---: | ---: | ---: | ---: |
+| hubbard `propagate` | 1,001,661,534 | 169 | 59.5 | 7.5 |
+| pauli `propagate` | 985,970,588 | 102 | 85.6 | 10.7 |
+| random `propagate` | 948,937,993 | 48.4 | 160 | 20.2 |
+| random `build_graph` | 948,937,993 | 53.2 | 174 | 21.8 |
+| random `energy` | 948,937,993 | 10.4 | 131 | 16.4 |
+| random `gradient` | 948,937,993 | 30.9 | 139 | 17.4 |
+| Schrödinger `build_graph` | 597,445,055 | 67.3 | 77.5 | 9.8 |
+| Schrödinger `energy` | 597,445,055 | 11.7 | 81.3 | 10.2 |
+| Schrödinger `gradient` | 597,445,055 | 39.5 | 102 | 12.8 |
+
+Measured at `N`=1, `R`=8, `P`=16 (medians of two reps, cell order flipped between reps). Term
+counts are identical to L2a's and L3's, which is the geometry-independence check.
+
+**`~GiB/node` is the sum over the node's ranks and `~GiB worst rank` is the largest single rank**,
+the same two columns L3 carries. The sum bounds the node and errs high — shared pages are charged
+to every rank — so it is the number to provision against and the wrong number to divide. In
+particular, reading L2b's 174 against L2a's 109 as "1.6x the memory" is a sum-against-max
+comparison; L2a's figure is one process's exact peak.
+
+What the pair does support is a per-rank reading, because peak RSS reproduces here where wall time
+does not: every memory cell above repeated across the two reps to within 0.2 GiB, and a separate
+one-rank run of the Heisenberg rows landed within 0.3 GiB of L2a's table on all three. Subtracting
+L2a's peak from the node sum and dividing by the eight ranks gives what each extra process costs
+beyond its share of the operator:
+
+| | Heisenberg | Schrödinger |
+| --- | ---: | ---: |
+| `build_graph` | 8.1 GiB/rank | 0.6 GiB/rank |
+| `energy` | 3.6 GiB/rank | 0.3 GiB/rank |
+| `gradient` | 3.6 GiB/rank | 0.2 GiB/rank |
+
+An order of magnitude apart on one shape, one node and one binary. The two flag sets differ in the
+observable — 14.75M terms against 200k — so what does not partition is what the observable drives,
+which is the same effect the L3 section records from the other direction. The raw observable is
+well under a GiB, so this is not the terms themselves being copied; something indexed by them
+scales with it. That has not been attributed further here.
+
+The consequence for sizing: **L2b is the memory-worst rung for the Heisenberg rows.** L2a holds one
+process, and L3 has more nodes to spread the same ranks over; L2b pays the per-rank cost `R` times
+on a single node. Size a job from the row and the rung being run, never from L2a alone.
+
 ## L3 — several nodes, the same problems
 
-The L2 problems again, at your own `N`, `R` and `P`. Choose `R × P` equal to L2's core count and
-the pair isolates what is paid per process rather than per core; choose anything else and it is
-still a valid multi-node run, just not that comparison.
+The L2b shape at `N` > 1. Hold `R` and `P` at exactly the values L2b ran, and the pair isolates
+the network: everything else about the two runs — cores per node, processes per node, partitions
+per process, flags, `-k` — is identical. Change `R` or `P` as well and it is still a valid
+multi-node run, just not that comparison.
 
-Same flags and same `-k` as L2, so only the measured cells differ. Term counts are
-geometry-independent (`cpp/tests/partition_equivalence_tests.cpp`), which is why one calibration
-serves every shape and a term-count disagreement means the *problem* changed, not the topology.
+Same flags and same `-k` as L2a, so only the measured cells differ — which is why one calibration
+of the sizes serves every rung.
 
 | row | terms | ~s | ~GiB/node | ~GiB worst rank |
 | --- | ---: | ---: | ---: | ---: |
@@ -202,7 +294,8 @@ serves every shape and a term-count disagreement means the *problem* changed, no
 | Schrödinger `gradient` | 597,445,055 | 8.9 | 36 | 4.5 |
 
 Measured at `N`=4, `R`=8, `P`=16 (medians of two reps) — a ballpark for sizing a job at that
-shape, not a target. Term counts are identical to L2's, which is the geometry-independence check.
+shape, not a target. Term counts are identical to L2a's and L2b's, which is the
+geometry-independence check.
 
 Memory per node does not fall the way time does, but there is **no fixed per-rank floor**: the
 Heisenberg rows pay 13.1 GiB on their worst rank and the Schrödinger rows 3.3 GiB, on the same
@@ -216,8 +309,8 @@ Heisenberg arm's 14.75M-term observable and every rank on the node held a copy o
 contributed nothing to the evolved state.
 
 The single-node budget, not this shape, is what sizes the Schrödinger rung: at `N`=4 its worst rank
-holds 4.5 GiB and the row would fit many times over. L2 and L3 share one flag set so the term
-counts stay comparable, so L2 is the binding constraint.
+holds 4.5 GiB and the row would fit many times over. L2a, L2b and L3 share one flag set so the
+term counts stay comparable, so L2a is the binding constraint.
 
 ## L4 — strong and weak scaling
 
@@ -267,19 +360,20 @@ was skipped.
 
 ## Shapes
 
-One rank over a whole node — L1 (with `P`=1) and L2:
+One rank over a whole node — L1 (with `P`=1) and L2a:
 
 ```bash
 #!/bin/bash
 #SBATCH --nodes=1 --ntasks-per-node=1 --cpus-per-task=128 --exclusive --mem=0 --time=0:30:00
 set -euo pipefail
 export monoprop_PARTITIONS=128 monoprop_NUM_THREADS=128
-just bench L2-hubbard-branch --hubbard-cutoff=10 --hubbard-lower-atol=3.38e-06 \
+just bench L2a-hubbard-branch --hubbard-cutoff=10 --hubbard-lower-atol=3.38e-06 \
     -k "test_model_propagate and hubbard"
 ```
 
-`N` nodes at `R` ranks × `P` partitions — L3 and L4. This is the same command with `srun` inserted
-and the report moved out of it; substitute your own `N`, `R` and `P` in all four places:
+`N` nodes at `R` ranks × `P` partitions — L2b (with `--nodes=1`), L3 and L4. This is the same
+command with `srun` inserted and the report moved out of it; substitute your own `N`, `R` and `P`
+in all four places:
 
 ```bash
 #!/bin/bash
@@ -288,7 +382,7 @@ set -euo pipefail
 export monoprop_PARTITIONS=16 monoprop_NUM_THREADS=16
 export monoprop_BENCH_LABEL=L3-hubbard-branch monoprop_BENCH_RESULTS=benches/results
 srun --ntasks-per-node=8 --cpus-per-task=16 --cpu-bind=cores \
-    uv run --no-sync python -m pytest benches -o filterwarnings=default \
+    .venv/bin/python -m pytest benches -o filterwarnings=default \
     --benchmark-json="benches/results/time-$monoprop_BENCH_LABEL.json" \
     -k "test_model_propagate and hubbard" \
     --hubbard-cutoff=10 --hubbard-lower-atol=3.38e-06
@@ -300,6 +394,10 @@ file. The two JSON artifacts are safe because only rank 0 writes them (`conftest
 `pytest_configure`). A ladder is a loop over sizes inside one allocation sized for its largest
 point.
 
+The interpreter is named directly rather than run through `uv run`, which re-resolves the
+environment in every rank — a shared-filesystem metadata storm at L4's rank counts. `just bench`
+is still the right entry point at one rank, where there is one resolution.
+
 ## Traps
 
 Each of these produces a plausible wrong number rather than an error.
@@ -307,6 +405,15 @@ Each of these produces a plausible wrong number rather than an error.
 - **`srun --cpu-bind=cores` with no `--cpus-per-task` on the `srun` confines each task to one
   core.** Measured in one allocation: no flags → 128 CPUs, `--cpus-per-task=128` → 128,
   `--cpu-bind=cores` alone → 1. Repeat both flags on the `srun`, not only on the `#SBATCH`.
+- **`P` above the cores the rank can see disables pinning entirely**, at 13.8x to 24.6x. This is
+  the mistake going from L2a to L2b invites: L2a's `monoprop_PARTITIONS=128` left in the
+  environment of an 8-rank launch asks each rank for 128 partitions inside a 16-core mask, and
+  `CpuTopology.cpp` then returns an empty placement order rather than refusing. `P` is `cores / R`
+  at L2b, not L2a's `P`. The rule is `P > visible cores`, full stop — not divisibility, which was
+  refuted at `P`=192 and `P`=256 where the ratio is exactly integer and both still collapsed. The
+  only warning goes to C++ stderr, which pytest's fd capture eats without `-s`. Read it back
+  instead: `meta.pinning.affinity_cpus_min/max` is the mask each rank got and
+  `single_cpu_threads_min/max` is how many threads landed on one core each.
 - **`--cpu-bind=none` cost 1.45x** against Slurm's own cgroup pinning at 8 ranks per node. With one
   rank holding a whole node there is nothing to confine and it stops mattering.
 - **`nproc` lies inside a job** — it honours `OMP_NUM_THREADS`, which the BLAS pinning sets to 1,
diff --git a/docs/content/docs/benchmarks.mdx b/docs/content/docs/benchmarks.mdx
index 859185ca..f4762d1f 100644
--- a/docs/content/docs/benchmarks.mdx
+++ b/docs/content/docs/benchmarks.mdx
@@ -306,9 +306,11 @@ monoprop_PARTITIONS=2 monoprop_NUM_THREADS=2 just bench-mpi r5t2 5 --map-by slot
 
 ### Benchmarking ladder
 
-`benches/LADDER.md` holds the ladder used for large-scale MPI benchmarking during development: four
+`benches/LADDER.md` holds the ladder used for large-scale MPI benchmarking during development: five
 groups of configurations, the flags and `-k` selector that produce each row, and the shapes to run
-them at.
+them at. Each rung differs from the one below it in exactly one thing -- partitions (L1 to L2a),
+processes (L2a to L2b), the network (L2b to L3), then node count (L3 to L4) -- so a regression that
+appears at one rung and not the one below it names its own cause.
 
 ### Continuous benchmarking
 

From 6ed7e7f06f7da0d1be6c3dba5ead01d2a4876fe2 Mon Sep 17 00:00:00 2001
From: Aaron Miller <61472721+diagonal-hamiltonian@users.noreply.github.com>
Date: Tue, 1 Sep 2026 13:58:35 +0100
Subject: [PATCH 11/14] Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Aaron Miller <61472721+diagonal-hamiltonian@users.noreply.github.com>
---
 .../monoprop-bench-tools/src/monoprop_bench_tools/models.py     | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/packages/monoprop-bench-tools/src/monoprop_bench_tools/models.py b/packages/monoprop-bench-tools/src/monoprop_bench_tools/models.py
index 0a5832fb..d23f2ac2 100644
--- a/packages/monoprop-bench-tools/src/monoprop_bench_tools/models.py
+++ b/packages/monoprop-bench-tools/src/monoprop_bench_tools/models.py
@@ -349,7 +349,7 @@ def build_hubbard_problem(
     """
     config = config or HubbardConfig()
     _check_modes(config.num_qubits, "--hubbard-num-sites")
-    # A lattice position, not a constant: off the lattice it silently changes the light cone.
+    # A lattice position, not a constant: shrinking the lattice without scaling shifts the light cone; out of range now raises.
     if not 0 <= config.observable_site < config.num_sites:
         msg = (
             f"--hubbard-observable-site {config.observable_site} is outside the "

From db323bd6f13eb63d5712f3adf16d2415b2c35ec4 Mon Sep 17 00:00:00 2001
From: Aaron Miller <aaron.miller@algorithmiq.fi>
Date: Tue, 1 Sep 2026 14:02:36 +0100
Subject: [PATCH 12/14] =?UTF-8?q?style(bench-tools):=20=F0=9F=8E=A8=20brin?=
 =?UTF-8?q?g=20the=20observable-site=20comment=20under=20the=20line=20limi?=
 =?UTF-8?q?t?=
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The autofix commit ran the line to 127 columns and closed it on "out of range now
raises", which the ValueError three lines down already states. Dropping that clause
keeps what the comment is for -- the site is a lattice position, so shrinking the
lattice without scaling it moves the light cone -- and fits 120.

Assisted-by: ClaudeCode:claude-opus-5
---
 .../monoprop-bench-tools/src/monoprop_bench_tools/models.py     | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/packages/monoprop-bench-tools/src/monoprop_bench_tools/models.py b/packages/monoprop-bench-tools/src/monoprop_bench_tools/models.py
index d23f2ac2..a95c1c36 100644
--- a/packages/monoprop-bench-tools/src/monoprop_bench_tools/models.py
+++ b/packages/monoprop-bench-tools/src/monoprop_bench_tools/models.py
@@ -349,7 +349,7 @@ def build_hubbard_problem(
     """
     config = config or HubbardConfig()
     _check_modes(config.num_qubits, "--hubbard-num-sites")
-    # A lattice position, not a constant: shrinking the lattice without scaling shifts the light cone; out of range now raises.
+    # A lattice position, not a constant: shrinking the lattice without scaling it shifts the light cone.
     if not 0 <= config.observable_site < config.num_sites:
         msg = (
             f"--hubbard-observable-site {config.observable_site} is outside the "

From a124cbef3c8a3d63170065862cc1f5982db9ac27 Mon Sep 17 00:00:00 2001
From: Aaron Miller <aaron.miller@algorithmiq.fi>
Date: Tue, 1 Sep 2026 18:06:06 +0100
Subject: [PATCH 13/14] less wordy

---
 benches/LADDER.md                | 533 ++++++++++---------------------
 benches/README.md                |   5 +-
 benches/conftest.py              |   8 +-
 docs/content/docs/benchmarks.mdx |   5 -
 4 files changed, 172 insertions(+), 379 deletions(-)

diff --git a/benches/LADDER.md b/benches/LADDER.md
index 2889101b..22bec49a 100644
--- a/benches/LADDER.md
+++ b/benches/LADDER.md
@@ -1,16 +1,35 @@
 # Comprehensive Benchmarking
 
-## Summary
+This benchmark separates kernel, shared-memory, process, network, and scaling effects. Define:
 
-| Group | What it measures |
-| --- | --- |
-| L1  | Single thread, 20M terms, 5GB |
-| L2a | Single-node perf, default user settings without MPI `1xcores`-shape. 0.95B terms Heisenberg, 0.60B Schrödinger, both ≤110GB|
-| L2b | The L2a problems on one node at `R × P` — MPI with no network in it |
-| L3  | Multi-node, same problem as L2b|
-| L4  | Strong/weak scaling. If L3 looks okay, don't bother running |
+- `N`: nodes
+- `C`: usable cores per node
+- `R`: MPI ranks per node
+- `P`: in-process partitions per rank
+- `P_total = N × R × P`: total partitions
 
-## Running one row
+Choose `R` for the platform, normally one rank per NUMA domain, then set `P = C / R`. Keep `R`
+and `P` fixed wherever the ladder requires a direct comparison.
+
+> **Example platform.** All flags, term counts, timings, and memory figures below were measured on
+> code based on `main` commit `97f95f762dfc7174243ccd59dd4ecbbb9775b610`, on Deucalion x86
+> nodes with `C=128`, using `R=8` and `P=16` for MPI runs. They are calibration and job-sizing
+> examples, not portable targets but memory should be relatively similar.
+
+## Ladder
+
+| rung | shape | purpose |
+| --- | --- | --- |
+| L1 | `N=1, R=1, P=1` | isolate single-thread kernel performance |
+| L2a | `N=1, R=1, P=C` | measure full-node shared-memory performance |
+| L2b | `N=1, R=R_platform, P=C/R` | isolate process and MPI overhead without a network hop |
+| L3 | `N>1`, same `R` and `P` as L2b | isolate the network |
+| L4 | vary `N`, hold `R` and `P` fixed | measure strong and weak scaling |
+
+L2a, L2b, and L3 use the same model flags. Only the execution shape changes. Term counts are
+topology-independent, so a mismatch means the problem definition changed.
+
+## Running a row
 
 ```bash
 uv sync --all-groups --all-extras          # once
@@ -18,307 +37,134 @@ just bench L1-hubbard --hubbard-cutoff=10 --hubbard-lower-atol=4.2e-05 \
     -k "test_model_propagate and hubbard"
 ```
 
-`LABEL` is one column in `benches/results/REPORT.md`, so name it for the group, the row and the
-arm: `L1-hubbard-branch`, `L1-hubbard-base`. Each run writes `results/time-<label>.json`
-(timings) and `results/<label>.json` (term counts, peak RSS, operator memory), then rebuilds the
-report. Reps are repeated invocations under different labels; leave `--bench-rounds` at 1, since
-`pedantic` builds round *k+1*'s arguments before releasing round *k*'s and a second round holds two
-propagators.
+Name `LABEL` by rung, row, and arm, for example `L1-hubbard-branch`. A run writes
+`results/time-<label>.json`, `results/<label>.json`, and rebuilds `benches/results/REPORT.md`.
+Use separate labels for repetitions and leave `--bench-rounds=1`; later rounds can overlap the
+previous round's live propagator and inflate memory.
 
-`build_graph`, `energy` and `gradient` may be selected together in one process: the timed
-`build_graph` publishes the graph it built and the two functionals evaluate that one, so the
-process builds a single operator. Keep `propagate` out of that selection — it holds its own
-operator, so adding it to the other three holds two per rank.
+Run `build_graph`, `energy`, and `gradient` together when all three are needed. They share one
+graph. Run `propagate` separately because it owns another operator.
 
 ## Declaring the shape
 
-Nodes `N`, ranks per node `R`, partitions per rank `P`, one core
-per partition — so `R × P` is the cores per node and `P_total` = `N × R × P` is what a
-communication cost is a function of. **`N` and `R` come from the launcher; `P` you must export.**
+The launcher sets `N` and `R`; export `P` explicitly:
 
 ```bash
-export monoprop_PARTITIONS=16 monoprop_NUM_THREADS=16
+export monoprop_PARTITIONS=<partitions-per-rank>
+export monoprop_NUM_THREADS=<cores-per-rank>
 ```
 
-Above one rank this is mandatory: `benches/conftest.py` refuses to start a multi-rank session with
-`monoprop_PARTITIONS` unset, because the engine's own default is `ranks == 1 ? cores : 1` and an
-unset knob would silently measure one partition per rank at a plausible wall time.
+For multi-rank runs this is mandatory. The engine otherwise defaults to one partition per rank.
+Before comparing results, check `ranks`, `nodes`, `ranks_per_node`, `partitions_env`, and
+`monoprop_threads` in the report's Configuration table.
 
-The rungs fix the shape as follows, so that each edge of the ladder moves one thing:
+## Calibrating model size
 
-| rung | `N` | `R` | `P` | the edge below it |
-| --- | ---: | ---: | ---: | --- |
-| L1 | 1 | 1 | 1 | — |
-| L2a | 1 | 1 | cores | in-process partitioning |
-| L2b | 1 | yours | cores / `R` | per process, no network |
-| L3 | yours | L2b's | L2b's | the network |
-| L4 | 1…64 | L2b's | L2b's | scaling in `N` |
+| model | preferred size control | constraint |
+| --- | --- | --- |
+| Hubbard | `--hubbard-lower-atol` | `2 × num_sites <= MAX_NUM_MODES` |
+| Pauli | `--pauli-lower-atol` | the supplied heavy-hex model requires 127 qubits |
+| random, Heisenberg | `--obs-terms` | duplicates make this an upper bound; fix `--seed` |
+| random, Schrödinger | `--num-generators` | also sets gradient length and work |
 
-The shape is recorded with every run — `ranks`, `nodes`, `ranks_per_node`, `partitions_env` and
-`monoprop_threads` in `results/<label>.json`'s `meta`, surfaced as the Configuration table of
-`REPORT.md`. Check it before comparing two rows: a ladder whose points did not all run at the same
-`R` and `P` is not a ladder.
 
-## Model size knobs
+Calibrate each row on the target platform before running the ladder:
 
-Every size below is an input, not a constant. The fixed models take `--<model>-<field>` for any
-field of their config dataclass (`just bench --help` lists them); the random problem takes
-`--num-generators`, `--num-modes`, `--cutoff`, `--obs-terms`, `--gen-length` and `--seed`.
+1. Fix the model structure and seed.
+2. Sweep the preferred size control until the full row fits the desired memory budget.
+3. Size graph rows against the largest peak of `build_graph`, `energy`, and `gradient`, not the
+   graph build alone.
+4. Reuse the chosen flags unchanged in L2a, L2b, and L3.
 
-| knob | range | note |
-| --- | --- | --- |
-| `--num-modes` | ≤ this build's `MAX_NUM_MODES` | rejected above it, rather than failing inside the extension |
-| `--hubbard-num-sites` | 2 × sites ≤ `MAX_NUM_MODES` | saturated from 60 at the default `lower_atol`; size Hubbard with `--hubbard-lower-atol` |
-| `--hubbard-observable-site` | `< --hubbard-num-sites` | a lattice position, not a constant: the default 46 sits 46/60 along the lattice, and shrinking the lattice without moving it changes the light cone. Out of range now raises |
-| `--pauli-num-qubits` | 127 only | `HEAVY_HEX_TOPOLOGY` is the fixed IBM Eagle coupling map, so any other count either indexes past the operator or leaves qubits uncoupled. Raises. Size this model with `--pauli-lower-atol` |
-| `--obs-terms` | any | an upper bound on the observable's terms, not an exact count: monomials are drawn independently and duplicates collapse, by about `obs_terms / 2·C(2·num_modes, gen_length)` — 0.06% at L1's 295k, 2.8% at L2a's 14.75M. Deterministic for a fixed `--seed`. **Heisenberg only** — see *Sizing the Schrödinger rows* |
-| `--num-generators` | any | the circuit's gate count, and the only usable size axis in the Schrödinger picture. It is also the length of the parameter vector, so `gradient` at 5800 generators does nearly six times the per-term work it does at 1000 |
-| `--gen-length` | `k % 4` in {0, 1} | a product of `k` Majoranas satisfies `(g_1...g_k)^dag = (-1)^(k(k-1)/2) g_1...g_k`, so it is Hermitian with the real coefficients drawn here only at those lengths. Verified over k=2..11: 4, 5, 8, 9 build and 2, 3, 6, 7, 10, 11 raise. Out of range now raises naming the option, rather than surfacing as `Non-Hermitian coeffs detected` from inside the extension |
-| `--pare-threshold` | any, default off | edge-retention cutoff for the graph functionals (`energy`, `gradient`). Unset keeps the exact graph |
-
-### Sizing the Schrödinger rows
-
-The Schrödinger picture evolves the state, so the observable does not drive the operator:
-`--obs-terms` is flat there (18.85M / 18.75M / 18.83M across a 74× sweep) and sizes the Heisenberg
-rows only. Of the remaining axes, two are cliffs rather than dials — one step of either goes from a
-few GiB to not fitting a 242 GiB node, because what grows is a transient inside `build_graph` and
-not the retained operator:
-
-| axis | last value that fits | next value tried |
-| --- | --- | --- |
-| `--cutoff` | 6 — 18,853,861 terms, 3.6 GiB | 7 — SIGKILL |
-| `--num-modes` | 160 — 29,082,430 terms, 5.6 GiB | 180 — SIGKILL |
-
-`--num-generators` is the axis that works. At `--cutoff=6 --num-modes=142 --obs-terms=200000`,
-one node at 1×128. Term counts are geometry- and operation-independent; the time and memory
-columns are not, so each row says what was measured — `build` is `build_graph` alone, `row` is all
-three operations in one process, which is what has to fit:
-
-| `--num-generators` | terms | ~s | ~GiB | measured |
-| ---: | ---: | ---: | ---: | --- |
-| 1000 | 18,853,861 | 1.3 | 3.6 | build |
-| 2000 | 24,707,186 | 3.2 | 4.2 | build |
-| 3000 | 42,566,127 | 6.1 | 7.2 | build |
-| 4000 | 97,631,757 | 13.3 | 16.2 | build |
-| 5000 | 261,929,069 | 30.6 | 32.2 | build |
-| 5600 | 485,959,982 | 97.9 | 78.0 | row |
-| **5800** | **597,445,055** | **115.9** | **99.7** | **row** |
-| 6000 | 737,159,992 | 142.8 | 115.1 | row |
-| 6200 | 907,357,991 | 172.1 | 143.1 | row |
-| 6400 | 1,115,757,089 | 117.0 | 138.5 | build |
-| 8000 | — | — | > 242 (SIGKILL) | build |
-
-L2a, L2b and L3 take the 5800-generator row: it is the largest whose whole-row peak fits the
-~110 GiB one-node budget the other L2a rows are held to. 6000 (115.1 GiB) and 6200 (143.1 GiB) clear 1B terms
-at 6400 but not the budget.
-
-**Size this rung against `gradient`, not `build_graph`.** At 6200 generators `build_graph` peaks at
-102.7 GiB and looks like it fits, while `gradient` in the same process peaks at 143.1 GiB. Two
-things make the last operation the expensive one, and only the second is about the gradient:
-
-- The three operations share a process, and peak RSS follows capacity rather than live bytes, so an
-  operation starts at the high-water mark its predecessors left. Measured at 6200: `gradient`'s
-  floor was 114.31 GiB against `energy`'s peak of 114.32 — the same bytes, not new ones. Of that
-  143.1 GiB row peak only 28.8 GiB is `gradient`'s own transient (`opmemdelta`).
-- That transient still tracks the *circuit*, not the operator: 28.8 GiB at 6200 generators against
-  7.9 GiB at 1000, on an operator that is slightly smaller (907M terms against 949M). Sizing this
-  picture by gate count therefore buys terms and gradient memory together.
-
-The axis accelerates — ×1.31, ×1.72, ×2.29, ×2.68 per extra 1000 gates — so read a size off this
-table rather than interpolating, and do not size the rung from two points: 1000→2000 alone
-predicts ~73,000 generators for 100M terms, where 4000 already reaches it. The counts are
-deterministic at a fixed `--seed`; 4000 reproduced to the term across two jobs.
-
-**A gate count is not a free parameter here.** `--num-generators` is also the length of the
-parameter vector, so the 5800-generator Schrödinger rows return a 5800-long gradient where the
-1000-generator Heisenberg rows return a 1000-long one. The two pictures' `gradient` cells are
-matched in terms and in memory, not in the work the functional does; do not read one against the
-other as a per-term ratio.
-
-## L1 — one thread
-
-One node, one rank, one partition: `export monoprop_PARTITIONS=1 monoprop_NUM_THREADS=1`.
-Removes every threading and MPI effect, so what moves is the kernel.
-
-The two `propagate` rows sit at ~10M terms and the two `gradient` rows at ~20M. The gradient rows
-are the same problem twice, once against the exact graph and once against a plan pared at
-`1e-10`, so a change to the functionals or to the masked plan shows on the pair rather than on
-either alone.
-
-| row | flags | `-k` | terms | ~s | ~GiB |
+In the Schrödinger picture, sweep number of qubits or `--num-generators` instead.
+
+## Example problem set
+
+Examples 128-core platform.
+
+### L1 example
+
+Set `monoprop_PARTITIONS=1` and `monoprop_NUM_THREADS=1`.
+
+| row | flags | selector | terms | time | peak RSS |
 | --- | --- | --- | ---: | ---: | ---: |
-| hubbard `propagate` | `--hubbard-cutoff=10 --hubbard-lower-atol=4.2e-05` | `test_model_propagate and hubbard` | 9,953,109 | 30 | 1 |
-| pauli `propagate` | `--pauli-cutoff=12 --pauli-lower-atol=1.22e-04` | `test_model_propagate and pauli` | 10,069,308 | 20 | 1 |
-| random `gradient` | `--num-generators=1000 --num-modes=142 --cutoff=6 --obs-terms=295000` | `test_random_gradient and heisenberg` | 19,902,244 | 11 | 2.5 |
-| random `gradient`, pared | *(same)* `--pare-threshold=1e-10` | `test_random_gradient and heisenberg` | 19,902,244 | 0.8 | 2.8 |
+| Hubbard `propagate` | `--hubbard-cutoff=10 --hubbard-lower-atol=4.2e-05` | `test_model_propagate and hubbard` | 9.95M | 30 s | 1 GiB |
+| Pauli `propagate` | `--pauli-cutoff=12 --pauli-lower-atol=1.22e-04` | `test_model_propagate and pauli` | 10.07M | 20 s | 1 GiB |
+| random `gradient` | `--num-generators=1000 --num-modes=142 --cutoff=6 --obs-terms=295000` | `test_random_gradient and heisenberg` | 19.90M | 11 s | 2.5 GiB |
+| random `gradient`, pared | same, plus `--pare-threshold=1e-10` | `test_random_gradient and heisenberg` | 19.90M | 0.8 s | 2.8 GiB |
+
+### L2a example
 
-## L2a — one node, one rank, ~1B terms
+Set `monoprop_PARTITIONS=C` and `monoprop_NUM_THREADS=C`. On the example platform, `C=128`. These are medians of three repetitions.
 
-One node, one rank, every core: `export monoprop_PARTITIONS=128 monoprop_NUM_THREADS=128`
-(substitute your core count). No MPI, so `P` alone carries the parallelism — this is the shape a
-user who never launches a rank gets, and the one whose partition count the engine would have
-picked on its own. The random rows are the only ones carrying all four operations, so they are
-where the operations separate.
 
-| row | flags | `-k` | terms | ~s | ~GiB |
+| row | flags | selector | terms | time | peak RSS |
 | --- | --- | --- | ---: | ---: | ---: |
-| hubbard `propagate` | `--hubbard-cutoff=10 --hubbard-lower-atol=3.38e-06` | `test_model_propagate and hubbard` | 1,001,661,534 | 140 | 60 |
-| pauli `propagate` | `--pauli-cutoff=14 --pauli-lower-atol=8.9e-06` | `test_model_propagate and pauli` | 985,970,588 | 90 | 80 |
-| random `propagate` | `--num-generators=1000 --num-modes=142 --cutoff=6 --obs-terms=14750000` | `test_random_propagate and heisenberg` | 948,937,993 | 50 | 95 |
-| random `build_graph` | *(same)* | `test_random_build_graph and heisenberg` | 948,937,993 | 50 | 109 |
-| random `energy` | *(same)* | `test_random_energy and heisenberg` | 948,937,993 | 9.9 | 102 |
-| random `gradient` | *(same)* | `test_random_gradient and heisenberg` | 948,937,993 | 34 | 110 |
-| Schrödinger `build_graph` | `--num-generators=5800 --num-modes=142 --cutoff=6 --obs-terms=200000` | `test_random_build_graph and schrodinger` | 597,445,055 | 66 | 73 |
-| Schrödinger `energy` | *(same)* | `test_random_energy and schrodinger` | 597,445,055 | 11 | 79 |
-| Schrödinger `gradient` | *(same)* | `test_random_gradient and schrodinger` | 597,445,055 | 39 | 100 |
-
-Medians of three reps, arm order flipped between reps. `energy` and `gradient` are separate rows
-because they are not one cost: `gradient` is 3.4x `energy` in the Heisenberg picture and 3.4x in
-the Schrödinger one.
-
-These nine rows are the problem set for L2b and L3 as well; only the shape changes.
-
-`build_graph`, `energy` and `gradient` share one process and one build — the graph the timed
-`build_graph` produces is what the other two evaluate — so run them with one selector, not three.
-`propagate` holds its own operator and stays out of that group.
-
-**The Schrödinger rows carry their own flags**, differing only in the gate count. `--obs-terms`
-cannot size that picture, so the rung is set by `--num-generators=5800` (see *Sizing the Schrödinger
-rows*), which is the largest gate count whose whole-row peak stays inside the ~110 GiB budget the
-Heisenberg rows sit at. That budget sizes the *flags*, which every rung then shares — but it does
-not bound the other rungs' memory, and measurement says L2b's is higher: per-rank overhead is paid
-`R` times and `N`=1 gives it no extra nodes to spread over. Size an L2b or L3 job from its own
-table. Two consequences worth reading before comparing the pictures:
-
-- The gradient's parameter vector is one entry per generator, so the Schrödinger `gradient`
-  differentiates 5800 parameters against the Heisenberg row's 1000. Unequal work, not just unequal
-  size.
-- Memory per term is higher in the Schrödinger picture — 0.167 GiB/Mterm at the `gradient` peak
-  against the Heisenberg rows' 0.115 — which is why matching the two pictures on memory leaves them
-  at 597M against 949M terms rather than at equal counts.
-
-## L2b — one node, `R` ranks × `P` partitions
-
-The L2a problems again, on one node, with the cores split across processes instead of held by one:
-`R × P` = the core count and `N` = 1. L2a → L2b is therefore the *per-process* step with no network
-in it, and L2b → L3 is the network step. Before this rung existed the ladder went from one rank
-straight to several nodes and moved both at once, which left anything that appeared at L3
-attributable to neither.
-
-Same flags and same `-k` as L2a — only the shape differs. Term counts are geometry-independent
-(`cpp/tests/partition_equivalence_tests.cpp`), so a term count that disagrees with L2a's means the
-*problem* changed, not the topology.
-
-### Choosing `R` and `P`
-
-Two constraints, and then one choice:
-
-- **`R × P` = the cores per node.** This is what makes the rung a comparison rather than a separate
-  measurement: the core count is held, so the only thing separating it from L2a is how many
-  processes hold those cores. Any other product is a valid run, just not this rung.
-- **`P` ≤ the cores one rank can see**, which under `--cpu-bind=cores` is its `--cpus-per-task`.
-  Above that the engine places nothing at all, at 13.8x to 24.6x, and says so only on C++ stderr —
-  see *Traps*. Note that this is the constraint L2a's `P` violates if it is left in the
-  environment: `P` here is `cores / R`.
-- **One rank per NUMA domain**, which then fixes both numbers on a given machine. The node these
-  tables were measured on is 8 domains × 16 cores, so `R`=8 and `P`=16.
-
-The third is a choice rather than a measured optimum, made for two reasons. It is the layout the
-library ships in, and it is **L3's shape** — L3 runs the same `R` and `P` at `N` > 1, so the two
-rungs differ in the node count and nothing else. If you change the split, change it in both rungs
-or the comparison they exist for is gone.
-
-| row | terms | ~s | ~GiB/node | ~GiB worst rank |
-| --- | ---: | ---: | ---: | ---: |
-| hubbard `propagate` | 1,001,661,534 | 169 | 59.5 | 7.5 |
-| pauli `propagate` | 985,970,588 | 102 | 85.6 | 10.7 |
-| random `propagate` | 948,937,993 | 48.4 | 160 | 20.2 |
-| random `build_graph` | 948,937,993 | 53.2 | 174 | 21.8 |
-| random `energy` | 948,937,993 | 10.4 | 131 | 16.4 |
-| random `gradient` | 948,937,993 | 30.9 | 139 | 17.4 |
-| Schrödinger `build_graph` | 597,445,055 | 67.3 | 77.5 | 9.8 |
-| Schrödinger `energy` | 597,445,055 | 11.7 | 81.3 | 10.2 |
-| Schrödinger `gradient` | 597,445,055 | 39.5 | 102 | 12.8 |
-
-Measured at `N`=1, `R`=8, `P`=16 (medians of two reps, cell order flipped between reps). Term
-counts are identical to L2a's and L3's, which is the geometry-independence check.
-
-**`~GiB/node` is the sum over the node's ranks and `~GiB worst rank` is the largest single rank**,
-the same two columns L3 carries. The sum bounds the node and errs high — shared pages are charged
-to every rank — so it is the number to provision against and the wrong number to divide. In
-particular, reading L2b's 174 against L2a's 109 as "1.6x the memory" is a sum-against-max
-comparison; L2a's figure is one process's exact peak.
-
-What the pair does support is a per-rank reading, because peak RSS reproduces here where wall time
-does not: every memory cell above repeated across the two reps to within 0.2 GiB, and a separate
-one-rank run of the Heisenberg rows landed within 0.3 GiB of L2a's table on all three. Subtracting
-L2a's peak from the node sum and dividing by the eight ranks gives what each extra process costs
-beyond its share of the operator:
-
-| | Heisenberg | Schrödinger |
-| --- | ---: | ---: |
-| `build_graph` | 8.1 GiB/rank | 0.6 GiB/rank |
-| `energy` | 3.6 GiB/rank | 0.3 GiB/rank |
-| `gradient` | 3.6 GiB/rank | 0.2 GiB/rank |
-
-An order of magnitude apart on one shape, one node and one binary. The two flag sets differ in the
-observable — 14.75M terms against 200k — so what does not partition is what the observable drives,
-which is the same effect the L3 section records from the other direction. The raw observable is
-well under a GiB, so this is not the terms themselves being copied; something indexed by them
-scales with it. That has not been attributed further here.
-
-The consequence for sizing: **L2b is the memory-worst rung for the Heisenberg rows.** L2a holds one
-process, and L3 has more nodes to spread the same ranks over; L2b pays the per-rank cost `R` times
-on a single node. Size a job from the row and the rung being run, never from L2a alone.
-
-## L3 — several nodes, the same problems
-
-The L2b shape at `N` > 1. Hold `R` and `P` at exactly the values L2b ran, and the pair isolates
-the network: everything else about the two runs — cores per node, processes per node, partitions
-per process, flags, `-k` — is identical. Change `R` or `P` as well and it is still a valid
-multi-node run, just not that comparison.
-
-Same flags and same `-k` as L2a, so only the measured cells differ — which is why one calibration
-of the sizes serves every rung.
-
-| row | terms | ~s | ~GiB/node | ~GiB worst rank |
-| --- | ---: | ---: | ---: | ---: |
-| hubbard `propagate` | 1,001,661,534 | 35 | 18 | |
-| pauli `propagate` | 985,970,588 | 23 | 25 | |
-| random `propagate` | 948,937,993 | 13 | 100 | |
-| random `build_graph` | 948,937,993 | 14 | 105 | 13.1 |
-| random `energy` | 948,937,993 | 2.3 | 61 | 7.7 |
-| random `gradient` | 948,937,993 | 7.6 | 63 | 7.9 |
-| Schrödinger `build_graph` | 597,445,055 | 20 | 26 | 3.3 |
-| Schrödinger `energy` | 597,445,055 | 1.9 | 26 | 3.3 |
-| Schrödinger `gradient` | 597,445,055 | 8.9 | 36 | 4.5 |
-
-Measured at `N`=4, `R`=8, `P`=16 (medians of two reps) — a ballpark for sizing a job at that
-shape, not a target. Term counts are identical to L2a's and L2b's, which is the
-geometry-independence check.
-
-Memory per node does not fall the way time does, but there is **no fixed per-rank floor**: the
-Heisenberg rows pay 13.1 GiB on their worst rank and the Schrödinger rows 3.3 GiB, on the same
-shape. An earlier revision of this table quoted a ~9.5 GiB per-rank floor "paid whatever the
-problem", which the Schrödinger rows here refute by a factor of four. Size a job from the row
-you are running.
-
-The Schrödinger rows are also where the flag split pays off: 26 GiB/node here against the
-80 GiB/node an earlier revision recorded at 18.8M terms, at 32x the terms. That row inherited the
-Heisenberg arm's 14.75M-term observable and every rank on the node held a copy of it, while it
-contributed nothing to the evolved state.
-
-The single-node budget, not this shape, is what sizes the Schrödinger rung: at `N`=4 its worst rank
-holds 4.5 GiB and the row would fit many times over. L2a, L2b and L3 share one flag set so the
-term counts stay comparable, so L2a is the binding constraint.
-
-## L4 — strong and weak scaling
-
-Hubbard `propagate` only: a communication cost is a function of `P_total`, so varying the model as
-well would confound it. Hold `R` and `P` fixed across every point of a ladder and vary only `N`;
-`P_total` is then `N × R × P`. The tables below were built at `R`=8, `P`=16.
-
-Size is set by `--hubbard-lower-atol` at a fixed `--hubbard-cutoff=10`.
+| Hubbard `propagate` | `--hubbard-cutoff=10 --hubbard-lower-atol=3.38e-06` | `test_model_propagate and hubbard` | 1.002B | 140 s | 60 GiB |
+| Pauli `propagate` | `--pauli-cutoff=14 --pauli-lower-atol=8.9e-06` | `test_model_propagate and pauli` | 0.986B | 90 s | 80 GiB |
+| random Heisenberg `propagate` | `--num-generators=1000 --num-modes=142 --cutoff=6 --obs-terms=14750000` | `test_random_propagate and heisenberg` | 0.949B | 50 s | 95 GiB |
+| random Heisenberg `build_graph` | same | `test_random_build_graph and heisenberg` | 0.949B | 50 s | 109 GiB |
+| random Heisenberg `energy` | same | `test_random_energy and heisenberg` | 0.949B | 9.9 s | 102 GiB |
+| random Heisenberg `gradient` | same | `test_random_gradient and heisenberg` | 0.949B | 34 s | 110 GiB |
+| random Schrödinger `build_graph` | `--num-generators=5800 --num-modes=142 --cutoff=6 --obs-terms=200000` | `test_random_build_graph and schrodinger` | 0.597B | 66 s | 73 GiB |
+| random Schrödinger `energy` | same | `test_random_energy and schrodinger` | 0.597B | 11 s | 79 GiB |
+| random Schrödinger `gradient` | same | `test_random_gradient and schrodinger` | 0.597B | 39 s | 100 GiB |
+
+
+## L2b: process overhead
+
+Run the L2a problems on one node with `R × P = C`. Use the same `R` and `P` in L3. One rank per
+NUMA domain is a portable starting point, not a guaranteed optimum.
+
+Example results at `N=1, R=8, P=16`:
+
+| row | time | RSS sum per node | worst-rank RSS |
+| --- | ---: | ---: | ---: |
+| Hubbard `propagate` | 169 s | 59.5 GiB | 7.5 GiB |
+| Pauli `propagate` | 102 s | 85.6 GiB | 10.7 GiB |
+| random Heisenberg `propagate` | 48.4 s | 160 GiB | 20.2 GiB |
+| random Heisenberg `build_graph` | 53.2 s | 174 GiB | 21.8 GiB |
+| random Heisenberg `energy` | 10.4 s | 131 GiB | 16.4 GiB |
+| random Heisenberg `gradient` | 30.9 s | 139 GiB | 17.4 GiB |
+| random Schrödinger `build_graph` | 67.3 s | 77.5 GiB | 9.8 GiB |
+| random Schrödinger `energy` | 11.7 s | 81.3 GiB | 10.2 GiB |
+| random Schrödinger `gradient` | 39.5 s | 102 GiB | 12.8 GiB |
+
+The node sum is an upper bound because shared pages are charged to every rank. Use it for
+provisioning, but do not compare it directly with a single process's peak. Per-rank overhead is
+model-dependent, so size L2b from its own measurements.
+
+## L3: network overhead
+
+Run the L2b shape on multiple nodes without changing `R`, `P`, model flags, or selectors. The
+L2b-to-L3 difference then isolates the network.
+
+Example results at `N=4, R=8, P=16`:
+
+| row | time | RSS sum per node | worst-rank RSS |
+| --- | ---: | ---: | ---: |
+| Hubbard `propagate` | 35 s | 18 GiB | — |
+| Pauli `propagate` | 23 s | 25 GiB | — |
+| random Heisenberg `propagate` | 13 s | 100 GiB | — |
+| random Heisenberg `build_graph` | 14 s | 105 GiB | 13.1 GiB |
+| random Heisenberg `energy` | 2.3 s | 61 GiB | 7.7 GiB |
+| random Heisenberg `gradient` | 7.6 s | 63 GiB | 7.9 GiB |
+| random Schrödinger `build_graph` | 20 s | 26 GiB | 3.3 GiB |
+| random Schrödinger `energy` | 1.9 s | 26 GiB | 3.3 GiB |
+| random Schrödinger `gradient` | 8.9 s | 36 GiB | 4.5 GiB |
+
+## L4: strong and weak scaling
+
+Use one representative operation and hold `R` and `P` fixed.
+
+- **Strong scaling:** hold the problem fixed and vary `N`.
+- **Weak scaling:** hold work per node approximately fixed while varying `N`.
+
+For strong scaling relative to the smallest run `(N0, t0)`, efficiency is
+`N0 × t0 / (N × t(N))`. For weak scaling relative to one node, it is `t(1) / t(N)`.
+
+The example uses Hubbard `propagate` with `--hubbard-cutoff=10`. This platform-specific sweep
+approximately doubles the term count at each step:
 
 | k | `--hubbard-lower-atol` | terms |
 | ---: | --- | ---: |
@@ -334,93 +180,52 @@ Size is set by `--hubbard-lower-atol` at a fixed `--hubbard-cutoff=10`.
 | 9 | `2.94e-07` | 48,317,129,677 |
 | 10 | `1.82e-07` | 94,684,031,363 |
 
-**Weak** — hold terms per node constant: **step one row down the table each time you double the
-node count.** Three ladders, each seven points from N=1 to N=64:
-
-| ladder | starts at | Mterms/node | ~s at N=1 → N=64 |
-| --- | --- | ---: | --- |
-| ~97M/node | k=0 at N=1 | 97 | 11 → 37 |
-| ~385M/node | k=2 at N=1 | 385 | 48 → 80 |
-| ~1529M/node | k=4 at N=1 | 1529 | 216 → 257 |
-
-That spread **is** the weak-scaling result; ideal scaling would be flat. Efficiency is
-`t(N=1)/t(N)`.
-
-**Strong** — hold one k and vary N. Three ladders:
+For a weak-scaling series that doubles `N`, advance one `k` per point. For strong scaling, choose
+one `k` that fits the smallest node count and keep it fixed. On another platform, build a fresh
+size sweep rather than treating this table as a target.
 
-| ladder | k | terms | N | ~s at smallest N → N=64 |
-| --- | ---: | ---: | --- | --- |
-| 1.57e9 fixed | 4 | 1,569,152,761 | 1…64 | 216 → 24 |
-| 6.13e9 fixed | 6 | 6,125,805,627 | 2…64 | 450 → 37 |
-| 2.44e10 fixed | 8 | 24,419,998,198 | 8…64 | 471 → 80 |
+## Launch templates
 
-Efficiency against the ladder's smallest node count `N0` is `N0 · t(N0) / (N · t(N))`. The larger
-two ladders start above one node because the problem does not fit below it, not because the point
-was skipped.
-
-## Shapes
-
-One rank over a whole node — L1 (with `P`=1) and L2a:
+For L1 and L2a, request one node and one rank. Set the core and partition counts from the granted
+CPU affinity:
 
 ```bash
-#!/bin/bash
-#SBATCH --nodes=1 --ntasks-per-node=1 --cpus-per-task=128 --exclusive --mem=0 --time=0:30:00
-set -euo pipefail
-export monoprop_PARTITIONS=128 monoprop_NUM_THREADS=128
+export monoprop_PARTITIONS="$C"
+export monoprop_NUM_THREADS="$C"
 just bench L2a-hubbard-branch --hubbard-cutoff=10 --hubbard-lower-atol=3.38e-06 \
     -k "test_model_propagate and hubbard"
 ```
 
-`N` nodes at `R` ranks × `P` partitions — L2b (with `--nodes=1`), L3 and L4. This is the same
-command with `srun` inserted and the report moved out of it; substitute your own `N`, `R` and `P`
-in all four places:
+For L2b, L3, and L4, request `R` ranks per node and `P` cores per rank. This Slurm example assumes
+the scheduler variables have already been set consistently:
 
 ```bash
-#!/bin/bash
-#SBATCH --nodes=4 --ntasks-per-node=8 --cpus-per-task=16 --exclusive --mem=0 --time=1:00:00
-set -euo pipefail
-export monoprop_PARTITIONS=16 monoprop_NUM_THREADS=16
-export monoprop_BENCH_LABEL=L3-hubbard-branch monoprop_BENCH_RESULTS=benches/results
-srun --ntasks-per-node=8 --cpus-per-task=16 --cpu-bind=cores \
+export monoprop_PARTITIONS="$P"
+export monoprop_NUM_THREADS="$P"
+export monoprop_BENCH_LABEL=L3-hubbard-branch
+export monoprop_BENCH_RESULTS=benches/results
+
+srun --ntasks-per-node="$R" --cpus-per-task="$P" --cpu-bind=cores \
     .venv/bin/python -m pytest benches -o filterwarnings=default \
     --benchmark-json="benches/results/time-$monoprop_BENCH_LABEL.json" \
     -k "test_model_propagate and hubbard" \
     --hubbard-cutoff=10 --hubbard-lower-atol=3.38e-06
+
 uv run --no-sync monoprop-bench-report benches/results
 ```
 
-The report runs **outside** `srun`: under it, one process per rank would race to write the same
-file. The two JSON artifacts are safe because only rank 0 writes them (`conftest.py`
-`pytest_configure`). A ladder is a loop over sizes inside one allocation sized for its largest
-point.
-
-The interpreter is named directly rather than run through `uv run`, which re-resolves the
-environment in every rank — a shared-filesystem metadata storm at L4's rank counts. `just bench`
-is still the right entry point at one rank, where there is one resolution.
-
-## Traps
-
-Each of these produces a plausible wrong number rather than an error.
-
-- **`srun --cpu-bind=cores` with no `--cpus-per-task` on the `srun` confines each task to one
-  core.** Measured in one allocation: no flags → 128 CPUs, `--cpus-per-task=128` → 128,
-  `--cpu-bind=cores` alone → 1. Repeat both flags on the `srun`, not only on the `#SBATCH`.
-- **`P` above the cores the rank can see disables pinning entirely**, at 13.8x to 24.6x. This is
-  the mistake going from L2a to L2b invites: L2a's `monoprop_PARTITIONS=128` left in the
-  environment of an 8-rank launch asks each rank for 128 partitions inside a 16-core mask, and
-  `CpuTopology.cpp` then returns an empty placement order rather than refusing. `P` is `cores / R`
-  at L2b, not L2a's `P`. The rule is `P > visible cores`, full stop — not divisibility, which was
-  refuted at `P`=192 and `P`=256 where the ratio is exactly integer and both still collapsed. The
-  only warning goes to C++ stderr, which pytest's fd capture eats without `-s`. Read it back
-  instead: `meta.pinning.affinity_cpus_min/max` is the mask each rank got and
-  `single_cpu_threads_min/max` is how many threads landed on one core each.
-- **`--cpu-bind=none` cost 1.45x** against Slurm's own cgroup pinning at 8 ranks per node. With one
-  rank holding a whole node there is nothing to confine and it stops mattering.
-- **`nproc` lies inside a job** — it honours `OMP_NUM_THREADS`, which the BLAS pinning sets to 1,
-  so a job holding 128 CPUs can log one core. Read `os.sched_getaffinity(0)`.
-- **Peak RSS follows capacity, not live bytes**, so a `shrink_to_fit` can raise it: `realloc` holds
-  the old and the new buffer at once.
-- **A two-operation row's peak is the MAX over its operations, never the sum.** `HighWaterMark`
-  resets `VmHWM` per benchmark, so both windows contain the same resident operator.
-- Unset `monoprop_PARTITIONS` and out-of-range model sizes used to be traps of this kind. Both now
-  raise; see *Declaring the shape* and *Model size knobs*.
+Run the report outside the MPI launcher. Invoke the virtual-environment interpreter directly on
+each rank to avoid repeated dependency resolution on shared filesystems.
+
+## Validation and common errors
+
+- Repeat the rank count, cores per rank, and binding options on the launcher command; scheduler
+  allocation directives may not propagate them.
+- Keep `P` no larger than the CPUs visible to each rank. Otherwise thread placement is disabled.
+- Use the scheduler affinity mask, such as `os.sched_getaffinity(0)`, to verify visible CPUs;
+  `nproc` may honor unrelated thread limits.
+- Keep CPU binding enabled unless the target platform has been measured otherwise.
+- Interpret peak RSS as a process high-water mark. Allocator capacity and temporary reallocation
+  can dominate live object size.
+- For grouped graph operations, compare the maximum peak across operations, not their sum.
+- Verify term counts and the recorded shape before interpreting timing differences.
\ No newline at end of file
diff --git a/benches/README.md b/benches/README.md
index d5106c4e..852d1ac2 100644
--- a/benches/README.md
+++ b/benches/README.md
@@ -42,10 +42,7 @@ roughly as `lower_atol^-1.9`.
 
 `--hubbard-observable-site` (default 46) is a lattice position, not a
 constant: sweeping `--hubbard-num-sites` without scaling it as 46/60 of the
-lattice slides the observable off-centre and changes the light cone. Off the
-lattice entirely it raises, as does a mode count above the extension's
-`MAX_NUM_MODES` and any `--pauli-num-qubits` other than the 127 that
-`HEAVY_HEX_TOPOLOGY` couples.
+lattice slides the observable off-centre and changes the light cone.
 
 `build_graph` extends the graph, so Hubbard's 29 Trotter steps retain 29
 layer-sets and exceed 229 GiB at 23.9M terms, where `propagate` runs the same
diff --git a/benches/conftest.py b/benches/conftest.py
index 7a56ec75..ec33dfe7 100644
--- a/benches/conftest.py
+++ b/benches/conftest.py
@@ -123,8 +123,7 @@ def _spread(comm: Any, value: int) -> dict[str, int]:
     return {"sum": _reduce_sum(comm, value), "max": _reduce_max(comm, value)}
 
 
-# (name, type, default, help). The type is per row because ``--pare-threshold`` is a float
-# that defaults to off, where every size knob is an int.
+# (name, type, default, help). 
 _RANDOM_OPTIONS = (
     ("gen-length", int, 4, "Majorana operators per generator."),
     ("obs-terms", int, 10000, "Observable terms."),
@@ -143,8 +142,6 @@ def _spread(comm: Any, value: int) -> dict[str, int]:
 _RESULTS: dict[str, Any] = {
     "meta": {},  # run configuration (ranks, threads, host, ...)
     "params": {},  # resolved random-problem hyperparameters
-    # Keyed by PYTEST node id, so a two-operation row gets one entry each and the windows
-    # overlap: max over them, never sum (each already contains the resident operator).
     "memhwm": {},  # pytest node id -> summed-over-ranks peak RSS, whole test, setup() included
     "memhwm_max": {},  # pytest node id -> worst-rank peak RSS, whole test, setup() included
     "opsize": {},  # picture / model / node id -> {"terms": n}
@@ -551,8 +548,7 @@ def built_graph(
 
     Session-scoped per picture so the graph is built once and shared across the
     read-only graph benchmarks (``energy``, ``gradient``), and records the operator
-    size for this picture while the graph is resident. A graph the timed
-    ``build_graph`` benchmark already published is reused instead of built again.
+    size for this picture while the graph is resident.
     """
     if picture in _GRAPH_CACHE:
         return _GRAPH_CACHE[picture]
diff --git a/docs/content/docs/benchmarks.mdx b/docs/content/docs/benchmarks.mdx
index f4762d1f..d72f679b 100644
--- a/docs/content/docs/benchmarks.mdx
+++ b/docs/content/docs/benchmarks.mdx
@@ -352,11 +352,6 @@ The bare-metal workflow provisions a dedicated node for each run. Its lower meas
 supports tighter timing thresholds and permits execution of the fixed models omitted on the shared
 runner.
 
-**No benchmark workflow gates a pull request.** Both track the main branch, and a threshold
-over a series that spans runners is a report, not a verdict. Measuring a change is the author's
-job -- see "Benchmarking ladder" above. Every run uploads `bmf.json` and the raw result JSON as an artifact,
-so a failed Bencher upload does not take the measurement with it.
-
 The `bench-markers` and `bench-rounds` inputs of the reusable `bench.yml` workflow select the
 benchmarks and repetition count. These values should remain fixed after establishing a testbed,
 because changing them changes the meaning of its time series:

From b4705843f03c8a5ffecdde454ef9a389b0a64af1 Mon Sep 17 00:00:00 2001
From: Aaron Miller <aaron.miller@algorithmiq.fi>
Date: Tue, 1 Sep 2026 19:12:06 +0100
Subject: [PATCH 14/14] =?UTF-8?q?style(benches):=20=F0=9F=8E=A8=20fix=20tr?=
 =?UTF-8?q?ailing=20whitespace=20and=20missing=20final=20newline?=
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Assisted-by: ClaudeCode:claude-opus-5
---
 benches/LADDER.md   | 2 +-
 benches/conftest.py | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/benches/LADDER.md b/benches/LADDER.md
index 22bec49a..36701a09 100644
--- a/benches/LADDER.md
+++ b/benches/LADDER.md
@@ -228,4 +228,4 @@ each rank to avoid repeated dependency resolution on shared filesystems.
 - Interpret peak RSS as a process high-water mark. Allocator capacity and temporary reallocation
   can dominate live object size.
 - For grouped graph operations, compare the maximum peak across operations, not their sum.
-- Verify term counts and the recorded shape before interpreting timing differences.
\ No newline at end of file
+- Verify term counts and the recorded shape before interpreting timing differences.
diff --git a/benches/conftest.py b/benches/conftest.py
index ec33dfe7..40ab612c 100644
--- a/benches/conftest.py
+++ b/benches/conftest.py
@@ -123,7 +123,7 @@ def _spread(comm: Any, value: int) -> dict[str, int]:
     return {"sum": _reduce_sum(comm, value), "max": _reduce_max(comm, value)}
 
 
-# (name, type, default, help). 
+# (name, type, default, help).
 _RANDOM_OPTIONS = (
     ("gen-length", int, 4, "Majorana operators per generator."),
     ("obs-terms", int, 10000, "Observable terms."),