feat(build)!: ✨ ship x86-64 wheels as a fat binary over five ISA tiers

AGENTS.md

@@ -40,6 +40,21 @@ monoprop is a high-performance C++/Python hybrid library implementing Majorana a
 - **Generated Code**: Python dispatch and C++ bindings auto-generated via `tools/generate-*.py`
 - **uv workspace**: the repository root is the `monoprop` package; `packages/*` holds the sibling
   distributions. See "Workspace layout" below.
+- **Tiered, not multiversioned**: the ISA is chosen per *whole library*, never per function.
+  An attribute cannot do this job: GCC will not inline across an `arch` mismatch (so a `target`-attributed
+  wrapper around the engine is a `jmp`, `flatten` notwithstanding) and `#pragma GCC target` does not
+  capture templates defined outside its region -- header-resident code is widened by its TU's command
+  line or not at all. And `flatten` + `target_clones` is unaffordable regardless: four flattened clones
+  of `build_layer`'s `with_algebra` x `with_store` x `with_kernel_width` fan-out took one TU from 16.7 s
+  to >20 min at ~100 GB of compiler memory. Consequences for the build: every engine
+  source goes through the `monoprop_engine_sources(...)` macro rather than
+  `target_sources(monoprop-objs ...)`, or it is missing from three of the four tiers; every per-target
+  setting goes through `_monoprop_configure_engine_objs` in `cpp/monoprop/CMakeLists.txt`, so the
+  tiers cannot drift apart in anything but arch flags. Consequence for numerics: `-ffp-contract=off`
+  is project-wide and is a **contract**, not a tuning knob -- without it `-march=x86-64-v3` and up
+  contract `a*b+c` into an FMA and the energy moves by 1-2 ULP, which in a fat binary means the same
+  wheel answering differently per host CPU. The byte-wise gate on it is a golden-baseline capture per
+  variant, which arrives with the baseline tooling.
 
 ### Workspace layout
 
@@ -90,6 +105,26 @@ Key files:
 - **Peak memory is the kernel's `VmHWM` high-water mark** — exact, with no sampling. Under
   MPI the ranks' peaks are summed, which errs high (disjoint transients, and shared pages
   charged to every rank): an upper bound, good for regressions, not for provisioning.
+- `cmake/compiler_flags/FatBinary.cmake`: the **only** place an ISA tier or a narrow-vector core is
+  declared, and it generates the loader's predicate table (`FatVariants.h`) so a tier cannot be built
+  without being selectable or selectable without being built. Five tiers, all `-mtune=skylake`:
+  `x86-64`, `-v2`, `-v3`, and `-v4 -mavx512vpopcntdq` at each of `-mprefer-vector-width=256` and `512`.
+  A published x86-64 wheel compiles the whole engine once per tier and `src/monoprop/_bootstrap.py`
+  loads one as `monoprop._core` at import, choosing with the tiny baseline-ISA probe
+  `src/monoprop/bindings/isa.cpp`. Off by default in source builds, where `-march=native` beats every
+  tier. See `docs/content/docs/fat-binary.mdx`.
+- **Two of the tiers are one ISA at two vector widths** (`...-vw256`, `...-vw512`): identical `-march`
+  and identical `__builtin_cpu_supports` requirements, differing only in `-mprefer-vector-width`. They
+  exist because GCC otherwise takes that from the `-mtune` tables, i.e. from `monoprop_FAT_MTUNE`, and
+  because no feature bit reports what the question actually turns on -- how wide the datapath behind the
+  registers is and what the core charges in clock for using it. So the discriminator is a core-name
+  table, `monoprop_FAT_NARROW_VECTOR_CORES`, read through `__builtin_cpu_is`; `znver4` is on it because
+  it is measured (1.1% to the narrow tier on the kicked-Ising model, disjoint ranges, *against* GCC's own
+  znver4 tuning). One consequence: each tier now carries **two** predicates -- `runnable` (features only,
+  what gates a `monoprop_VARIANT` pin) and `preferred` (plus the table, what the automatic selection and
+  `supported_variants()` read) -- and conflating them makes the wide tier unpinnable on exactly the
+  machines worth comparing it on. Nothing but a disassembly can tell the pair apart, so
+  `tests/test_variants.py` asserts that the width *setting* differs while the feature set does not.
 
 ### Core abstractions (the propagation backbone)
 
@@ -172,7 +207,9 @@ mp = MajoranaPropagator(operator, initial_state, cutoff=4)
 4. Use trailing return type syntax in function declarations.
 5. Add a one-line `///` summary if the declaration is in `cpp/include/monoprop/`; elsewhere add a plain
    `//` note only where the code does not already say it.
-6. Implement in the corresponding `.cpp` under `cpp/monoprop/`.
+6. Implement in the corresponding `.cpp` under `cpp/monoprop/`, and register a *new* `.cpp` with
+   `monoprop_engine_sources(...)` -- never `target_sources(monoprop-objs ...)`, which reaches only
+   the baseline fat-binary tier.
 7. Add Python bindings in `src/monoprop/bindings/binder.h`
 8. Regenerate bindings with `tools/generate-binders.py`
 9. Test with both C++ and Python tests

CMakeLists.txt

@@ -57,6 +57,16 @@ option(
 
 option(monoprop_ENABLE_CXX_UNIT_TESTS "Enable C++ unit test suite" ON)
 
+# Compile the engine once per x86-64 ISA tier and pick one when monoprop is imported. This is what
+# published wheels want and what a source build does not: a source build has -march=native, which is
+# strictly better than any tier, so the default is OFF. See cmake/compiler_flags/FatBinary.cmake and
+# docs/content/docs/fat-binary.mdx.
+option(
+  monoprop_ENABLE_FAT_BINARY
+  "Build one copy of the engine per x86-64 ISA tier and dispatch at import time"
+  OFF
+)
+
 set(Python_FIND_VIRTUALENV FIRST)
 find_package(
   Python
@@ -82,7 +92,7 @@ message(STATUS "Configuring a ${CMAKE_BUILD_TYPE} build")
 string(TOUPPER ${CMAKE_BUILD_TYPE} _cmake_build_type_upper)
 
 message(STATUS "Compiler flags for ${CMAKE_CXX_COMPILER_ID}")
-message(STATUS "   From environment       : ${CMAKE_CXX_FLAGS}")
+message(STATUS "   From environment       : ${monoprop_CXX_FLAGS_FROM_ENV}")
 set(
   _cmake_build_type_specific_flags
   "${CMAKE_CXX_FLAGS_${_cmake_build_type_upper}}"
@@ -92,6 +102,12 @@ message(
   "   Build-type-specific    : ${_cmake_build_type_specific_flags}"
 )
 message(STATUS "   Vectorization flag     : ${ARCH_FLAG}")
+message(STATUS "   Fat binary             : ${monoprop_ENABLE_FAT_BINARY}")
+if(monoprop_ENABLE_FAT_BINARY)
+  message(STATUS "   ISA tiers              : ${monoprop_FAT_TIERS}")
+  message(STATUS "   Tier tuning            : -mtune=${monoprop_FAT_MTUNE}")
+  message(STATUS "   Baseline ISA floor     : ${_monoprop_baseline_flags}")
+endif()
 message(
   STATUS
   "   Project defaults       : ${CMAKE_CXX${CMAKE_CXX_STANDARD}_STANDARD_COMPILE_OPTION} ${monoprop_CXX_FLAGS}"

README.md

@@ -46,6 +46,12 @@ pip install monoprop      # or: uv add monoprop
 The prebuilt PyPI wheels are single-process (built **without** MPI). For multi-rank
 runs, or to build the C++ library and executables, build from source (see below).
 
+The `x86-64` wheels are **fat binaries**: they carry the engine compiled for four
+instruction-set levels (`x86-64`, `x86-64-v2`, `x86-64-v3`, and `x86-64-v4` with
+`avx512vpopcntdq`), and pick the best one the CPU can execute when `monoprop` is
+imported. `monoprop.__variant__` says which one loaded; `monoprop_VARIANT` pins one.
+See the [fat-binary guide](https://docs.monoprop.algorithmiq.tech/fat-binary).
+
 ## Quick example
 
 Back-propagate a Majorana observable through a one-gate circuit:
@@ -98,6 +104,10 @@ uv sync --all-extras -v
 uv sync --all-extras -v --config-settings=cmake.define.monoprop_ENABLE_MPI=ON
 ```
 
+A source build compiles with `-march=native`, which is faster than any wheel and not
+portable off the build machine. The multi-ISA build the wheels use is off by default;
+`just build-fat` turns it on.
+
 C++ unit-test build:
 
 ```bash
@@ -117,6 +127,7 @@ uv sync --all-groups --all-extras -v    # installs the workspace, incl. the benc
 uv run python -m pytest -m "not mpi"   # Python tests (serial)
 just test-mpi                          # Python + C++ tests under MPI
 just test-wide                         # Python + C++ unit tests with a 64-bit TermIndex
+just test-variants                     # Python tests once per ISA variant (fat builds)
 ```
 
 See the [testing guide](https://docs.monoprop.algorithmiq.tech/testing)

cmake/compiler_flags/CXXFlags.cmake

@@ -53,6 +53,11 @@ cmake_dependent_option(
   OFF
 )
 
+# What CXXFLAGS actually contained, recorded before anything appends to CMAKE_CXX_FLAGS. The fat
+# binary appends its baseline ISA floor there (see FatBinary.cmake), and the status report has to
+# be able to tell the two apart or it attributes our flags to the user's environment.
+set(monoprop_CXX_FLAGS_FROM_ENV "${CMAKE_CXX_FLAGS}")
+
 # code needs C++23 at least
 set(CMAKE_CXX_STANDARD 23)
 set(CMAKE_CXX_STANDARD_REQUIRED ON)
@@ -68,16 +73,24 @@ set(CMAKE_POSITION_INDEPENDENT_CODE TRUE)
 set(CMAKE_CXX_VISIBILITY_PRESET "hidden")
 set(CMAKE_VISIBILITY_INLINES_HIDDEN TRUE)
 
+# The single place an architecture is chosen. Everything downstream reads monoprop_ARCH_MARCH rather
+# than deciding again from monoprop_ENABLE_ARCH_FLAGS, so that what is compiled and what is reported
+# cannot disagree -- a build that advertises an ISA it did not compile for makes every benchmark
+# artifact and every monoprop.__compiler_flags__ a guess.
+#
+# monoprop_ARCH_MARCH is the variant *id* a single-ISA build reports as monoprop.__variant__:
+# "native", or "default" for a build with no -march flag. The flags themselves are ARCH_FLAG, which is
+# what the provenance query reads.
 set(ARCH_FLAG "")
+set(monoprop_ARCH_MARCH "default")
 if(monoprop_ENABLE_ARCH_FLAGS)
-  if(CMAKE_CXX_COMPILER_ID MATCHES GNU)
-    set(ARCH_FLAG "-march=native")
-  endif()
-  if(CMAKE_CXX_COMPILER_ID MATCHES Clang)
+  if(CMAKE_CXX_COMPILER_ID MATCHES "GNU|Clang")
     set(ARCH_FLAG "-march=native")
+    set(monoprop_ARCH_MARCH "native")
   endif()
   if(CMAKE_CXX_COMPILER_ID MATCHES Intel)
     set(ARCH_FLAG "-xHost")
+    set(monoprop_ARCH_MARCH "native")
   endif()
 endif()
 
@@ -159,24 +172,62 @@ function(_monoprop_query_machine_flags)
   set(${_arg_OUTPUT_VARIABLE} "${_flags}" PARENT_SCOPE)
 endfunction()
 
-# Empty is the no-arch-flag build and queries the default target.
-set(monoprop_DEFAULT_VARIANT_FLAGS "")
-_monoprop_query_machine_flags(
-  ARCH_FLAGS ${ARCH_FLAG}
-  OUTPUT_VARIABLE monoprop_DEFAULT_VARIANT_FLAGS
-)
+# Write a Variants.h reporting one variant's identity and the machine flags the compiler actually
+# resolved for it. Called once per fat-binary tier, into a per-tier include directory that the tier's
+# object library puts ahead of the shared one, plus once for the plain single-ISA build.
+#
+# This is the fix for a provenance bug worth naming: the header used to be configured once, from a
+# query of -march=native whenever monoprop_ENABLE_ARCH_FLAGS was ON regardless of what was actually
+# compiled. So monoprop.__variant__, monoprop.__compiler_flags__ and every benchmark artifact's
+# machine-flags entry reported the host's ISA even when the build had been pointed somewhere else --
+# which is exactly the metadata a fat binary needs to be trustworthy, since it is how you tell which
+# tier got loaded.
+#
+# Usage:
+#   _monoprop_generate_variant_header(VARIANT_ID <id> OUTPUT_DIR <dir> [ARCH_FLAGS <flags...>])
+function(_monoprop_generate_variant_header)
+  cmake_parse_arguments(
+    PARSE_ARGV 0
+    _arg
+    ""
+    "VARIANT_ID;OUTPUT_DIR"
+    "ARCH_FLAGS"
+  )
 
-set(monoprop_VARIANTS "")
-set(monoprop_VARIANT_FLAGS "")
+  if(NOT _arg_VARIANT_ID OR NOT _arg_OUTPUT_DIR)
+    message(
+      FATAL_ERROR
+      "_monoprop_generate_variant_header: VARIANT_ID and OUTPUT_DIR are required"
+    )
+  endif()
+
+  # Unquoted on purpose: cmake_parse_arguments(PARSE_ARGV) escapes the semicolons inside a single
+  # argument, so a quoted list arrives as one flag spelled "-march=x86-64\;-mtune=skylake\;..." and
+  # the query silently reports the compiler's defaults instead of the variant's.
+  _monoprop_query_machine_flags(
+    ARCH_FLAGS ${_arg_ARCH_FLAGS}
+    OUTPUT_VARIABLE monoprop_VARIANT_MACHINE_FLAGS
+  )
+  set(monoprop_VARIANT_ID "${_arg_VARIANT_ID}")
 
-# generate a header file with the macros needed to describe the variant
-configure_file(
-  ${PROJECT_SOURCE_DIR}/cpp/include/monoprop/Variants.h.in
-  ${PROJECT_BINARY_DIR}/include/monoprop/Variants.h
-  @ONLY
+  configure_file(
+    ${PROJECT_SOURCE_DIR}/cpp/include/monoprop/Variants.h.in
+    ${_arg_OUTPUT_DIR}/monoprop/Variants.h
+    @ONLY
+  )
+endfunction()
+
+_monoprop_generate_variant_header(
+  VARIANT_ID "${monoprop_ARCH_MARCH}"
+  ARCH_FLAGS ${ARCH_FLAG}
+  OUTPUT_DIR "${PROJECT_BINARY_DIR}/include"
 )
 
 set(monoprop_CXX_FLAGS "")
 include(${CMAKE_CURRENT_LIST_DIR}/GNU.CXX.cmake)
 include(${CMAKE_CURRENT_LIST_DIR}/Intel.CXX.cmake)
 include(${CMAKE_CURRENT_LIST_DIR}/Clang.CXX.cmake)
+
+# Must come last: with the fat binary enabled this overwrites ARCH_FLAG with the baseline tier's flags
+# and defines the per-tier engine targets.
+include(${CMAKE_CURRENT_LIST_DIR}/FatBinary.cmake)

cmake/compiler_flags/Clang.CXX.cmake

@@ -16,9 +16,17 @@ if(CMAKE_CXX_COMPILER_ID MATCHES Clang)
     )
   endif()
 
+  # -ffp-contract=off, and why it is not a micro-optimization to be traded away: without it the
+  # compiler fuses a*b+c into an FMA wherever the target has one, which changes the rounding of the
+  # coefficient accumulation. Measured across ISA levels, every evolved term stays bit-identical and
+  # only the energy moves, by 1-2 ULP from -march=x86-64-v3 up. That is small and it is also exactly
+  # the wrong shape: with a fat binary the same wheel would answer differently depending on which CPU
+  # it landed on, and `just diff-baseline` could no longer be a byte-wise gate. The project has no
+  # -ffast-math and treats accumulation order as a contract, so contraction is off everywhere -- not
+  # only in the tiers -- to keep a source build, a wheel and every tier bit-comparable.
   set(
     monoprop_CXX_FLAGS
-    "-Wall -Wno-padded -Wno-unknown-pragmas -Woverloaded-virtual -Wwrite-strings -fcolor-diagnostics -Wno-c++98-compat -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer"
+    "-Wall -Wno-padded -Wno-unknown-pragmas -Woverloaded-virtual -Wwrite-strings -fcolor-diagnostics -Wno-c++98-compat -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -ffp-contract=off"
   )
   set(CMAKE_CXX_FLAGS_RELEASE "-O3 -DNDEBUG")
   set(