diff --git a/AGENTS.md b/AGENTS.md index cf2f2fd0..bf26dd24 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -40,6 +40,21 @@ monoprop is a high-performance C++/Python hybrid library implementing Majorana a - **Generated Code**: Python dispatch and C++ bindings auto-generated via `tools/generate-*.py` - **uv workspace**: the repository root is the `monoprop` package; `packages/*` holds the sibling distributions. See "Workspace layout" below. +- **Tiered, not multiversioned**: the ISA is chosen per *whole library*, never per function. + An attribute cannot do this job: GCC will not inline across an `arch` mismatch (so a `target`-attributed + wrapper around the engine is a `jmp`, `flatten` notwithstanding) and `#pragma GCC target` does not + capture templates defined outside its region -- header-resident code is widened by its TU's command + line or not at all. And `flatten` + `target_clones` is unaffordable regardless: four flattened clones + of `build_layer`'s `with_algebra` x `with_store` x `with_kernel_width` fan-out took one TU from 16.7 s + to >20 min at ~100 GB of compiler memory. Consequences for the build: every engine + source goes through the `monoprop_engine_sources(...)` macro rather than + `target_sources(monoprop-objs ...)`, or it is missing from three of the four tiers; every per-target + setting goes through `_monoprop_configure_engine_objs` in `cpp/monoprop/CMakeLists.txt`, so the + tiers cannot drift apart in anything but arch flags. Consequence for numerics: `-ffp-contract=off` + is project-wide and is a **contract**, not a tuning knob -- without it `-march=x86-64-v3` and up + contract `a*b+c` into an FMA and the energy moves by 1-2 ULP, which in a fat binary means the same + wheel answering differently per host CPU. The byte-wise gate on it is a golden-baseline capture per + variant, which arrives with the baseline tooling. ### Workspace layout @@ -90,6 +105,26 @@ Key files: - **Peak memory is the kernel's `VmHWM` high-water mark** — exact, with no sampling. Under MPI the ranks' peaks are summed, which errs high (disjoint transients, and shared pages charged to every rank): an upper bound, good for regressions, not for provisioning. +- `cmake/compiler_flags/FatBinary.cmake`: the **only** place an ISA tier or a narrow-vector core is + declared, and it generates the loader's predicate table (`FatVariants.h`) so a tier cannot be built + without being selectable or selectable without being built. Five tiers, all `-mtune=skylake`: + `x86-64`, `-v2`, `-v3`, and `-v4 -mavx512vpopcntdq` at each of `-mprefer-vector-width=256` and `512`. + A published x86-64 wheel compiles the whole engine once per tier and `src/monoprop/_bootstrap.py` + loads one as `monoprop._core` at import, choosing with the tiny baseline-ISA probe + `src/monoprop/bindings/isa.cpp`. Off by default in source builds, where `-march=native` beats every + tier. See `docs/content/docs/fat-binary.mdx`. +- **Two of the tiers are one ISA at two vector widths** (`...-vw256`, `...-vw512`): identical `-march` + and identical `__builtin_cpu_supports` requirements, differing only in `-mprefer-vector-width`. They + exist because GCC otherwise takes that from the `-mtune` tables, i.e. from `monoprop_FAT_MTUNE`, and + because no feature bit reports what the question actually turns on -- how wide the datapath behind the + registers is and what the core charges in clock for using it. So the discriminator is a core-name + table, `monoprop_FAT_NARROW_VECTOR_CORES`, read through `__builtin_cpu_is`; `znver4` is on it because + it is measured (1.1% to the narrow tier on the kicked-Ising model, disjoint ranges, *against* GCC's own + znver4 tuning). One consequence: each tier now carries **two** predicates -- `runnable` (features only, + what gates a `monoprop_VARIANT` pin) and `preferred` (plus the table, what the automatic selection and + `supported_variants()` read) -- and conflating them makes the wide tier unpinnable on exactly the + machines worth comparing it on. Nothing but a disassembly can tell the pair apart, so + `tests/test_variants.py` asserts that the width *setting* differs while the feature set does not. ### Core abstractions (the propagation backbone) @@ -172,7 +207,9 @@ mp = MajoranaPropagator(operator, initial_state, cutoff=4) 4. Use trailing return type syntax in function declarations. 5. Add a one-line `///` summary if the declaration is in `cpp/include/monoprop/`; elsewhere add a plain `//` note only where the code does not already say it. -6. Implement in the corresponding `.cpp` under `cpp/monoprop/`. +6. Implement in the corresponding `.cpp` under `cpp/monoprop/`, and register a *new* `.cpp` with + `monoprop_engine_sources(...)` -- never `target_sources(monoprop-objs ...)`, which reaches only + the baseline fat-binary tier. 7. Add Python bindings in `src/monoprop/bindings/binder.h` 8. Regenerate bindings with `tools/generate-binders.py` 9. Test with both C++ and Python tests diff --git a/CMakeLists.txt b/CMakeLists.txt index c98b529e..d344a46e 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -57,6 +57,16 @@ option( option(monoprop_ENABLE_CXX_UNIT_TESTS "Enable C++ unit test suite" ON) +# Compile the engine once per x86-64 ISA tier and pick one when monoprop is imported. This is what +# published wheels want and what a source build does not: a source build has -march=native, which is +# strictly better than any tier, so the default is OFF. See cmake/compiler_flags/FatBinary.cmake and +# docs/content/docs/fat-binary.mdx. +option( + monoprop_ENABLE_FAT_BINARY + "Build one copy of the engine per x86-64 ISA tier and dispatch at import time" + OFF +) + set(Python_FIND_VIRTUALENV FIRST) find_package( Python @@ -82,7 +92,7 @@ message(STATUS "Configuring a ${CMAKE_BUILD_TYPE} build") string(TOUPPER ${CMAKE_BUILD_TYPE} _cmake_build_type_upper) message(STATUS "Compiler flags for ${CMAKE_CXX_COMPILER_ID}") -message(STATUS " From environment : ${CMAKE_CXX_FLAGS}") +message(STATUS " From environment : ${monoprop_CXX_FLAGS_FROM_ENV}") set( _cmake_build_type_specific_flags "${CMAKE_CXX_FLAGS_${_cmake_build_type_upper}}" @@ -92,6 +102,12 @@ message( " Build-type-specific : ${_cmake_build_type_specific_flags}" ) message(STATUS " Vectorization flag : ${ARCH_FLAG}") +message(STATUS " Fat binary : ${monoprop_ENABLE_FAT_BINARY}") +if(monoprop_ENABLE_FAT_BINARY) + message(STATUS " ISA tiers : ${monoprop_FAT_TIERS}") + message(STATUS " Tier tuning : -mtune=${monoprop_FAT_MTUNE}") + message(STATUS " Baseline ISA floor : ${_monoprop_baseline_flags}") +endif() message( STATUS " Project defaults : ${CMAKE_CXX${CMAKE_CXX_STANDARD}_STANDARD_COMPILE_OPTION} ${monoprop_CXX_FLAGS}" diff --git a/README.md b/README.md index 3f03bbbe..60453d73 100644 --- a/README.md +++ b/README.md @@ -46,6 +46,12 @@ pip install monoprop # or: uv add monoprop The prebuilt PyPI wheels are single-process (built **without** MPI). For multi-rank runs, or to build the C++ library and executables, build from source (see below). +The `x86-64` wheels are **fat binaries**: they carry the engine compiled for four +instruction-set levels (`x86-64`, `x86-64-v2`, `x86-64-v3`, and `x86-64-v4` with +`avx512vpopcntdq`), and pick the best one the CPU can execute when `monoprop` is +imported. `monoprop.__variant__` says which one loaded; `monoprop_VARIANT` pins one. +See the [fat-binary guide](https://docs.monoprop.algorithmiq.tech/fat-binary). + ## Quick example Back-propagate a Majorana observable through a one-gate circuit: @@ -98,6 +104,10 @@ uv sync --all-extras -v uv sync --all-extras -v --config-settings=cmake.define.monoprop_ENABLE_MPI=ON ``` +A source build compiles with `-march=native`, which is faster than any wheel and not +portable off the build machine. The multi-ISA build the wheels use is off by default; +`just build-fat` turns it on. + C++ unit-test build: ```bash @@ -117,6 +127,7 @@ uv sync --all-groups --all-extras -v # installs the workspace, incl. the benc uv run python -m pytest -m "not mpi" # Python tests (serial) just test-mpi # Python + C++ tests under MPI just test-wide # Python + C++ unit tests with a 64-bit TermIndex +just test-variants # Python tests once per ISA variant (fat builds) ``` See the [testing guide](https://docs.monoprop.algorithmiq.tech/testing) diff --git a/cmake/compiler_flags/CXXFlags.cmake b/cmake/compiler_flags/CXXFlags.cmake index a6805df6..9eb2ea5f 100644 --- a/cmake/compiler_flags/CXXFlags.cmake +++ b/cmake/compiler_flags/CXXFlags.cmake @@ -53,6 +53,11 @@ cmake_dependent_option( OFF ) +# What CXXFLAGS actually contained, recorded before anything appends to CMAKE_CXX_FLAGS. The fat +# binary appends its baseline ISA floor there (see FatBinary.cmake), and the status report has to +# be able to tell the two apart or it attributes our flags to the user's environment. +set(monoprop_CXX_FLAGS_FROM_ENV "${CMAKE_CXX_FLAGS}") + # code needs C++23 at least set(CMAKE_CXX_STANDARD 23) set(CMAKE_CXX_STANDARD_REQUIRED ON) @@ -68,16 +73,24 @@ set(CMAKE_POSITION_INDEPENDENT_CODE TRUE) set(CMAKE_CXX_VISIBILITY_PRESET "hidden") set(CMAKE_VISIBILITY_INLINES_HIDDEN TRUE) +# The single place an architecture is chosen. Everything downstream reads monoprop_ARCH_MARCH rather +# than deciding again from monoprop_ENABLE_ARCH_FLAGS, so that what is compiled and what is reported +# cannot disagree -- a build that advertises an ISA it did not compile for makes every benchmark +# artifact and every monoprop.__compiler_flags__ a guess. +# +# monoprop_ARCH_MARCH is the variant *id* a single-ISA build reports as monoprop.__variant__: +# "native", or "default" for a build with no -march flag. The flags themselves are ARCH_FLAG, which is +# what the provenance query reads. set(ARCH_FLAG "") +set(monoprop_ARCH_MARCH "default") if(monoprop_ENABLE_ARCH_FLAGS) - if(CMAKE_CXX_COMPILER_ID MATCHES GNU) - set(ARCH_FLAG "-march=native") - endif() - if(CMAKE_CXX_COMPILER_ID MATCHES Clang) + if(CMAKE_CXX_COMPILER_ID MATCHES "GNU|Clang") set(ARCH_FLAG "-march=native") + set(monoprop_ARCH_MARCH "native") endif() if(CMAKE_CXX_COMPILER_ID MATCHES Intel) set(ARCH_FLAG "-xHost") + set(monoprop_ARCH_MARCH "native") endif() endif() @@ -159,24 +172,62 @@ function(_monoprop_query_machine_flags) set(${_arg_OUTPUT_VARIABLE} "${_flags}" PARENT_SCOPE) endfunction() -# Empty is the no-arch-flag build and queries the default target. -set(monoprop_DEFAULT_VARIANT_FLAGS "") -_monoprop_query_machine_flags( - ARCH_FLAGS ${ARCH_FLAG} - OUTPUT_VARIABLE monoprop_DEFAULT_VARIANT_FLAGS -) +# Write a Variants.h reporting one variant's identity and the machine flags the compiler actually +# resolved for it. Called once per fat-binary tier, into a per-tier include directory that the tier's +# object library puts ahead of the shared one, plus once for the plain single-ISA build. +# +# This is the fix for a provenance bug worth naming: the header used to be configured once, from a +# query of -march=native whenever monoprop_ENABLE_ARCH_FLAGS was ON regardless of what was actually +# compiled. So monoprop.__variant__, monoprop.__compiler_flags__ and every benchmark artifact's +# machine-flags entry reported the host's ISA even when the build had been pointed somewhere else -- +# which is exactly the metadata a fat binary needs to be trustworthy, since it is how you tell which +# tier got loaded. +# +# Usage: +# _monoprop_generate_variant_header(VARIANT_ID OUTPUT_DIR [ARCH_FLAGS ]) +function(_monoprop_generate_variant_header) + cmake_parse_arguments( + PARSE_ARGV 0 + _arg + "" + "VARIANT_ID;OUTPUT_DIR" + "ARCH_FLAGS" + ) -set(monoprop_VARIANTS "") -set(monoprop_VARIANT_FLAGS "") + if(NOT _arg_VARIANT_ID OR NOT _arg_OUTPUT_DIR) + message( + FATAL_ERROR + "_monoprop_generate_variant_header: VARIANT_ID and OUTPUT_DIR are required" + ) + endif() + + # Unquoted on purpose: cmake_parse_arguments(PARSE_ARGV) escapes the semicolons inside a single + # argument, so a quoted list arrives as one flag spelled "-march=x86-64\;-mtune=skylake\;..." and + # the query silently reports the compiler's defaults instead of the variant's. + _monoprop_query_machine_flags( + ARCH_FLAGS ${_arg_ARCH_FLAGS} + OUTPUT_VARIABLE monoprop_VARIANT_MACHINE_FLAGS + ) + set(monoprop_VARIANT_ID "${_arg_VARIANT_ID}") -# generate a header file with the macros needed to describe the variant -configure_file( - ${PROJECT_SOURCE_DIR}/cpp/include/monoprop/Variants.h.in - ${PROJECT_BINARY_DIR}/include/monoprop/Variants.h - @ONLY + configure_file( + ${PROJECT_SOURCE_DIR}/cpp/include/monoprop/Variants.h.in + ${_arg_OUTPUT_DIR}/monoprop/Variants.h + @ONLY + ) +endfunction() + +_monoprop_generate_variant_header( + VARIANT_ID "${monoprop_ARCH_MARCH}" + ARCH_FLAGS ${ARCH_FLAG} + OUTPUT_DIR "${PROJECT_BINARY_DIR}/include" ) set(monoprop_CXX_FLAGS "") include(${CMAKE_CURRENT_LIST_DIR}/GNU.CXX.cmake) include(${CMAKE_CURRENT_LIST_DIR}/Intel.CXX.cmake) include(${CMAKE_CURRENT_LIST_DIR}/Clang.CXX.cmake) + +# Must come last: with the fat binary enabled this overwrites ARCH_FLAG with the baseline tier's flags +# and defines the per-tier engine targets. +include(${CMAKE_CURRENT_LIST_DIR}/FatBinary.cmake) diff --git a/cmake/compiler_flags/Clang.CXX.cmake b/cmake/compiler_flags/Clang.CXX.cmake index 019d0330..26fc008d 100644 --- a/cmake/compiler_flags/Clang.CXX.cmake +++ b/cmake/compiler_flags/Clang.CXX.cmake @@ -16,9 +16,17 @@ if(CMAKE_CXX_COMPILER_ID MATCHES Clang) ) endif() + # -ffp-contract=off, and why it is not a micro-optimization to be traded away: without it the + # compiler fuses a*b+c into an FMA wherever the target has one, which changes the rounding of the + # coefficient accumulation. Measured across ISA levels, every evolved term stays bit-identical and + # only the energy moves, by 1-2 ULP from -march=x86-64-v3 up. That is small and it is also exactly + # the wrong shape: with a fat binary the same wheel would answer differently depending on which CPU + # it landed on, and `just diff-baseline` could no longer be a byte-wise gate. The project has no + # -ffast-math and treats accumulation order as a contract, so contraction is off everywhere -- not + # only in the tiers -- to keep a source build, a wheel and every tier bit-comparable. set( monoprop_CXX_FLAGS - "-Wall -Wno-padded -Wno-unknown-pragmas -Woverloaded-virtual -Wwrite-strings -fcolor-diagnostics -Wno-c++98-compat -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer" + "-Wall -Wno-padded -Wno-unknown-pragmas -Woverloaded-virtual -Wwrite-strings -fcolor-diagnostics -Wno-c++98-compat -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -ffp-contract=off" ) set(CMAKE_CXX_FLAGS_RELEASE "-O3 -DNDEBUG") set( diff --git a/cmake/compiler_flags/FatBinary.cmake b/cmake/compiler_flags/FatBinary.cmake new file mode 100644 index 00000000..8fe04bed --- /dev/null +++ b/cmake/compiler_flags/FatBinary.cmake @@ -0,0 +1,440 @@ +#.rst: +# +# The fat binary: one copy of the propagation engine per x86-64 ISA tier, all four in the same wheel, +# one of them selected when ``monoprop`` is imported. +# +# Why the ISA is bound on the command line and not by an attribute. Two things, both measured. +# +# An attribute cannot widen code defined outside it. ``target("arch=x86-64-v4,avx512vpopcntdq")`` is +# perfectly valid -- a comma separates options in a plain ``target``, unlike ``target_clones`` where it +# separates clones -- but ``ix86_can_inline_p`` refuses to inline across an ``arch`` mismatch, so a +# targeted wrapper around this engine compiles to a ``jmp``. ``flatten`` does not override that, and +# ``#pragma GCC target`` does not capture a template that was defined outside its region. Header-resident +# code is widened by its TU's command line or not at all. +# +# And ``flatten`` plus ``target_clones`` is not affordable here even so. Flattening pulls the layer +# engine's whole instantiation fan-out into one function body and the clones then multiply it: measured +# on the runtime-width engine, where that fan-out is one TU, four clones took it from 16.7 s to >20 min +# at ~100 GB of compiler memory -- 24.5 GB even with the widest axis collapsed, against 740 MB for the +# same code as an ordinary compile. As separate TUs the multiplication is linear and parallel; inside +# one function it is not. Here the fan-out is wider still, the engine being templated on the mode +# count. See ``docs/content/docs/fat-binary.mdx``. +# +# Why not glibc-hwcaps, which would need no code at all: its subdirectory names are the four psABI +# levels, and the top tier here is ``x86-64-v4`` *plus* ``avx512vpopcntdq``. Installing it as +# ``x86-64-v4`` would hand it to Skylake-X and Cascade Lake, which are v4 and have no vector popcount, +# and they would take SIGILL. The predicate has to be ours. +# +# Variables used:: +# +# monoprop_ENABLE_FAT_BINARY declared in the top-level CMakeLists.txt +# monoprop_FAT_MTUNE +# +# Variables defined:: +# +# monoprop_FAT_TIERS tier ids, baseline first +# monoprop_ENGINE_OBJ_TARGETS object libraries holding the engine, one per tier +# ARCH_FLAG overwritten with the baseline tier's flags +# +# Provides:: +# +# monoprop_engine_sources(...) add sources to every engine object library + +# What every tier tunes for. Not part of the ISA: -mtune never widens the instruction set, it only +# changes the cost model and the schedule, so it is free to name a core no tier requires. skylake is +# the measured choice -- it reaches the top cluster on every vectorization counter at the smallest +# .text, and is the oldest core in that cluster, so it is the least likely to schedule badly across the +# 2015-onwards range. There is no vendor-neutral alternative: GCC rejects -mtune=x86-64-v3. +set( + monoprop_FAT_MTUNE + "skylake" + CACHE STRING + "-mtune value applied to every fat-binary tier (scheduling only; never widens the ISA)" +) + +# Tier ids, baseline first. An id is the install directory name, the value monoprop.__variant__ +# reports and the value monoprop_VARIANT accepts, so it is user-visible and appears in benchmark +# artifacts: renaming one orphans whatever tracked those. +# +# The last two are one instruction set at two vector widths: identical -march, identical +# __builtin_cpu_supports requirements, differing only in -mprefer-vector-width. They are a pair because +# with -mprefer-vector-width unset GCC takes the AVX-512 vector width from the tuning tables, i.e. from +# monoprop_FAT_MTUNE -- a core chosen for its *schedule*, on the reasoning that -mtune never changes +# which instructions come out. For AVX-512 widths it does. So the two tiers pin it and the run time +# picks, using monoprop_FAT_NARROW_VECTOR_CORES below. Their order is load-bearing: the generated table +# is this list reversed, so the 512-bit tier is tried first and a narrow-datapath core falls through to +# the 256-bit one. +set( + monoprop_FAT_TIERS + "x86-64-v1" + "x86-64-v2" + "x86-64-v3" + "x86-64-v4-vpopcntdq-vw256" + "x86-64-v4-vpopcntdq-vw512" +) + +# Cores that should be given 256-bit vectors even though they can execute 512-bit ones. +# +# This is a table of core names and not a feature query because there is no feature to query. Whether +# 512-bit vectors are worth using is a property of the *implementation* -- how wide the datapath behind +# the registers really is, and what the core charges in clock frequency for lighting it up -- and CPUID +# reports neither. -mprefer-vector-width is a tuning flag for exactly that reason, and a build that +# fixes it has guessed on the user's behalf; carrying both and choosing at run time is the only way the +# answer can be right on more than one machine. +# +# A core is on the list when it has either of the two reasons to keep 512-bit code out, and off it when +# it has neither: +# +# a split datapath -- 512-bit operations run on 256-bit hardware, so the width halves the instruction +# count and buys no throughput. AMD Zen 4 is the case, and the one entry here that is measured rather +# than reasoned: pinning the two tiers on an EPYC 9R14, the 256-bit tier wins the 127-qubit +# kicked-Ising model by 1.1% with the two three-sample ranges disjoint (358.9-361.3 ms against +# 363.5-365.7) and ties on the 120-mode Hubbard one (2058.7 against 2063.2, spreads overlapping). GCC +# tunes znver4 the other way -- -mtune=znver4 resolves to width 512 -- so this is a correction to it. +# +# a frequency penalty -- the core drops its clock while 512-bit code is in flight. Ice Lake through +# Rocket Lake pay a measured ~175 MHz of peak, which is what put -mprefer-vector-width=256 in both +# compilers' Intel tuning to begin with. +# +# Not on the list: Zen 5, which has the full-width datapath Zen 4 lacks, and Sapphire Rapids onwards, +# where the frequency penalty went away (llvm/llvm-project#102047) -- a decision against GCC's own +# -mtune tables, which still say 256 for every Intel AVX-512 core. Neither can be measured here, so +# both are the mechanism argument rather than a number, and both are falsifiable with monoprop_VARIANT. +# +# Also not on the list, and not by choice: the parts the 256-bit default was introduced for in the first +# place. Skylake-SP, Cascade Lake and Cooper Lake are x86-64-v4 with no vector popcount, so they never +# reach this tier at all and their names would be dead entries. +# +# A name GCC does not recognise is dropped with a warning below rather than being an error: the cost is +# that that core gets the 512-bit tier, which is a percent or so, and the alternative is a build that +# fails on an older toolchain over a tuning hint. +set( + monoprop_FAT_NARROW_VECTOR_CORES + "znver4" + "icelake-client" + "icelake-server" + "tigerlake" + "rocketlake" +) + +# Resolve a tier id to the flags it compiles with, the CPU features it requires, and anything else its +# selection predicate needs. +# +# The first two are deliberately separate: MARCH_VAR is what the compiler is told, CPU_TOKENS_VAR is what +# the loader checks, and they are not the same list. -march=x86-64-v3 permits the compiler to use every +# v3 instruction, but __builtin_cpu_supports("x86-64-v3") is one query covering the whole level, so the +# token list is shorter than the flag list rather than being derived from it. +function(_monoprop_tier_spec) + set( + _one_value_args + TIER + MARCH_VAR + CPU_TOKENS_VAR + EXTRA_PREDICATE_VAR + ) + cmake_parse_arguments(PARSE_ARGV 0 _arg "" "${_one_value_args}" "") + + # Anything a __builtin_cpu_supports token cannot say. Empty for every tier but one. + set(_extra "") + + if(_arg_TIER STREQUAL "x86-64-v1") + set(_march "-march=x86-64") + set(_tokens "") + elseif(_arg_TIER STREQUAL "x86-64-v2") + set(_march "-march=x86-64-v2") + set(_tokens "x86-64-v2") + elseif(_arg_TIER STREQUAL "x86-64-v3") + set(_march "-march=x86-64-v3") + set(_tokens "x86-64-v3") + elseif(_arg_TIER MATCHES "^x86-64-v4-vpopcntdq-vw(256|512)$") + # v4 alone buys nothing here: ablating the eight AVX-512 extensions one at a time, + # -mavx512vpopcntdq accounted for the entire v4 -> v4x gain and the other seven for exactly zero. + # This codebase is std::popcount word loops and holds no intrinsics, so a vector popcount is the + # only extension it has anything to bite on. + # + # The width is spelled out rather than left to -mtune, and that is the point of the pair: + # -mtune=skylake and -mtune=znver4 resolve to 512, -mtune=icelake-server and -mtune=sapphirerapids + # to 256. See monoprop_FAT_TIERS above. + set(_width "${CMAKE_MATCH_1}") + set( + _march + "-march=x86-64-v4" + "-mavx512vpopcntdq" + "-mprefer-vector-width=${_width}" + ) + set( + _tokens + "x86-64-v4" + "avx512vpopcntdq" + ) + if(_width STREQUAL "512") + # The only tier with a predicate term that is not a feature bit, and the reason the 512 tier is + # tried first: a narrow-datapath core fails this and falls through to the 256 tier, which asks for + # nothing but the instructions. + set(_extra "!monoprop::detail::prefers_narrow_vectors()") + endif() + else() + message(FATAL_ERROR "_monoprop_tier_spec: unknown tier '${_arg_TIER}'") + endif() + + list(APPEND _march "-mtune=${monoprop_FAT_MTUNE}") + + # Reproducibility across tiers, and the reason it needs saying: without this, -march=x86-64-v3 and up + # contract a*b+c into FMA3, which changes the rounding of the coefficient accumulation. Measured: all + # evolved terms stay bit-identical but the energy moves by 1-2 ULP from v3 up, and an arm built with + # -ffp-contract=off came back byte-identical to the baseline. In a fat binary that would make a + # result depend on which CPU the user happens to run on, which is not something a propagation library + # gets to do -- and it would end `just diff-baseline` as a byte-wise gate. The project-wide flag in + # GNU.CXX.cmake already covers this; it is repeated here so a tier cannot lose it by reordering. + list(APPEND _march "-ffp-contract=off") + + if(_arg_MARCH_VAR) + set(${_arg_MARCH_VAR} "${_march}" PARENT_SCOPE) + endif() + if(_arg_CPU_TOKENS_VAR) + set(${_arg_CPU_TOKENS_VAR} "${_tokens}" PARENT_SCOPE) + endif() + if(_arg_EXTRA_PREDICATE_VAR) + set(${_arg_EXTRA_PREDICATE_VAR} "${_extra}" PARENT_SCOPE) + endif() +endfunction() + +# Sanitize a tier id into something usable as a CMake target-name suffix. +function(_monoprop_tier_slug tier output_variable) + string( + REPLACE "-" + "_" + _slug + "${tier}" + ) + set(${output_variable} "${_slug}" PARENT_SCOPE) +endfunction() + +if(NOT monoprop_ENABLE_FAT_BINARY) + set(monoprop_ENGINE_OBJ_TARGETS "monoprop-objs") + + macro(monoprop_engine_sources) + target_sources(monoprop-objs PRIVATE ${ARGN}) + endmacro() + + return() +endif() + +# --------------------------------------------------------------------------------------------------- +# From here on the fat binary is enabled. +# --------------------------------------------------------------------------------------------------- + +if(NOT CMAKE_SYSTEM_PROCESSOR MATCHES "^(x86_64|amd64|AMD64)$") + message( + FATAL_ERROR + "monoprop_ENABLE_FAT_BINARY is x86-64 only (CMAKE_SYSTEM_PROCESSOR is '${CMAKE_SYSTEM_PROCESSOR}'). Turn it OFF; a single-ISA build is the correct shape on this target." + ) +endif() + +if(NOT CMAKE_CXX_COMPILER_ID MATCHES "^(GNU|Clang|AppleClang)$") + message( + FATAL_ERROR + "monoprop_ENABLE_FAT_BINARY needs the -march/-mtune spelling and __builtin_cpu_supports, i.e. GCC or Clang. Detected '${CMAKE_CXX_COMPILER_ID}'." + ) +endif() + +# Which of the core names in monoprop_FAT_NARROW_VECTOR_CORES this compiler actually knows. Checked +# rather than assumed: __builtin_cpu_is rejects a name its libgcc has no model number for, and the name +# set grows with every GCC release, so a hard-coded list is a build failure waiting for the oldest +# toolchain in the matrix. A dropped name means that core takes the 512-bit tier instead of the 256-bit +# one -- a tuning difference, never a fault, since the two tiers require identical CPU features. +include(CheckCXXSourceCompiles) +set(_monoprop_narrow_cores "") +foreach(_core IN LISTS monoprop_FAT_NARROW_VECTOR_CORES) + string(MAKE_C_IDENTIFIER "monoprop_HAVE_CPU_IS_${_core}" _monoprop_have_core) + check_cxx_source_compiles( + "int main() { return __builtin_cpu_is(\"${_core}\"); }" + ${_monoprop_have_core} + ) + if(${_monoprop_have_core}) + list(APPEND _monoprop_narrow_cores "${_core}") + else() + message( + WARNING + "${CMAKE_CXX_COMPILER_ID} does not know the core name '${_core}', so __builtin_cpu_is cannot test for it. That CPU will be given the 512-bit vector tier instead of the 256-bit one; both require the same instructions, so this costs tuning and not correctness." + ) + endif() +endforeach() + +set(_monoprop_narrow_core_list "") +foreach(_core IN LISTS _monoprop_narrow_cores) + string( + APPEND _monoprop_narrow_core_list + " X(\"${_core}\") \\\n" + ) +endforeach() + +# The loader's variant table, best ISA first. Generated rather than hand-written in isa.cpp so that the +# tier list above is the only place a tier is declared: a tier that is built but never selected, or +# selected but never built, is a silent loss of the whole feature. +set(_monoprop_variant_table_body "") +list(REVERSE monoprop_FAT_TIERS) +foreach(_tier IN LISTS monoprop_FAT_TIERS) + _monoprop_tier_spec( + TIER "${_tier}" + CPU_TOKENS_VAR _tokens + EXTRA_PREDICATE_VAR _extra_predicate + ) + if(_tokens STREQUAL "") + set(_predicate "true") + else() + set(_predicate "") + foreach(_token IN LISTS _tokens) + if(NOT _predicate STREQUAL "") + string(APPEND _predicate " && ") + endif() + string(APPEND _predicate "__builtin_cpu_supports(\"${_token}\")") + endforeach() + endif() + # Two predicates, and the distinction is load-bearing: the first is whether this CPU *can execute* + # the tier, which is what gates a monoprop_VARIANT pin, and the second is whether it should be *given* + # the tier, which is what the automatic selection reads. They differ for exactly one tier -- the + # 512-bit one, which any AVX-512 CPU can run and only some should have -- and conflating them makes + # that tier unpinnable on the machines somebody would want to compare it on. + set(_preferred "${_predicate}") + if(NOT _extra_predicate STREQUAL "") + set(_preferred "${_predicate} && ${_extra_predicate}") + endif() + string( + APPEND _monoprop_variant_table_body + " X(\"${_tier}\", ${_predicate}, ${_preferred}) \\\n" + ) +endforeach() +list(REVERSE monoprop_FAT_TIERS) + +file( + WRITE "${PROJECT_BINARY_DIR}/include/monoprop/FatVariants.h" + "// Generated by cmake/compiler_flags/FatBinary.cmake -- do not edit. +#pragma once + +/// Cores that should be given the 256-bit-preferring variant, as X(core name). May be empty. +#define monoprop_FAT_NARROW_VECTOR_CORES(X) \\ +${_monoprop_narrow_core_list} /* end */ + +namespace monoprop::detail { +/// Whether this CPU should be handed 256-bit vectors in preference to 512-bit ones. +/// +/// Not a capability question -- both variants that ask it require exactly the same instructions -- but +/// an implementation one: how wide the datapath behind the registers is, and what the core charges in +/// clock frequency for using all of it. CPUID reports neither, so this is a name table, and a core +/// nobody listed gets the wider variant. See monoprop_FAT_NARROW_VECTOR_CORES in FatBinary.cmake. +inline auto prefers_narrow_vectors() -> bool { +#if defined(__x86_64__) || defined(_M_X64) + // Required before any other __builtin_cpu_* call in a translation unit that may run before libgcc's + // own constructor has. Idempotent, so callers that have already probed are not a problem. + __builtin_cpu_init(); +#define monoprop_FAT_NARROW_CORE(core) \\ + if (__builtin_cpu_is(core)) { \\ + return true; \\ + } + monoprop_FAT_NARROW_VECTOR_CORES(monoprop_FAT_NARROW_CORE) +#undef monoprop_FAT_NARROW_CORE +#endif + return false; +} +} // namespace monoprop::detail + +/// Every shipped ISA variant, best first, as X(id, runnable, preferred). +/// +/// The second holds when the CPU can execute the variant at all, the third when it should be given the +/// variant without being asked. They differ only for the 512-bit vector variant, and only because +/// vector width is a tuning question rather than a capability one -- see prefers_narrow_vectors(). +/// +/// Best-first is the selection order, so the table's order is load-bearing and not cosmetic. +#define monoprop_FAT_VARIANT_TABLE(X) \\ +${_monoprop_variant_table_body} /* end */ +" +) + +# The baseline tier is not a fifth build: it *is* monoprop-objs, and so also what libmonoprop.so, the +# C++ unit tests and any C++ consumer of the installed package get. That is deliberate -- a wheel's +# tiers are chosen for the machines that run it, while a source build has -march=native available and +# does not need any of this -- and it means CI's C++ tests exercise the portable floor. +list(GET monoprop_FAT_TIERS 0 monoprop_FAT_BASELINE_TIER) +_monoprop_tier_spec(TIER "${monoprop_FAT_BASELINE_TIER}" MARCH_VAR ARCH_FLAG) + +# The baseline is also the *floor*, and it has to be global rather than per target. monoprop-objs is +# not the only source of code in a wheel: nanobind's static library, and anything else a dependency +# adds as its own target, is compiled with whatever -march the toolchain defaults to -- and that +# default is not the psABI baseline. GCC as shipped by Ubuntu here is configured +# --with-arch-64=x86-64-v3, so without this floor the v1 and v2 variants would carry AVX2 in their +# nanobind glue, and _isa -- the probe that exists precisely to keep us off machines that cannot run +# a variant -- would itself fault on those machines. +# +# Putting it in CMAKE_CXX_FLAGS rather than on targets is what makes it a floor: it is emitted before +# every target's own options, so a tier's wider -march still wins for that tier's objects, while +# everything nobody widened stays at the baseline. +string( + REPLACE ";" + " " + _monoprop_baseline_flags + "${ARCH_FLAG}" +) +string(APPEND CMAKE_CXX_FLAGS " ${_monoprop_baseline_flags}") + +# One engine object library per tier, and one Variants.h per tier so each can say which it is. The +# baseline tier reuses monoprop-objs rather than adding a target, so N tiers cost N compiles and not +# N+1. +set(monoprop_ENGINE_OBJ_TARGETS "") +foreach(_tier IN LISTS monoprop_FAT_TIERS) + _monoprop_tier_spec(TIER "${_tier}" MARCH_VAR _tier_flags) + _monoprop_generate_variant_header( + VARIANT_ID "${_tier}" + ARCH_FLAGS ${_tier_flags} + OUTPUT_DIR "${PROJECT_BINARY_DIR}/variants/${_tier}/include" + ) + + if(_tier STREQUAL monoprop_FAT_BASELINE_TIER) + list(APPEND monoprop_ENGINE_OBJ_TARGETS "monoprop-objs") + else() + _monoprop_tier_slug("${_tier}" _slug) + add_library(monoprop-objs-${_slug} OBJECT "") + list(APPEND monoprop_ENGINE_OBJ_TARGETS "monoprop-objs-${_slug}") + endif() +endforeach() + +# The engine object library that carries a given tier, and the include directory holding that tier's +# Variants.h. Both are derived, so callers never spell a target name or a path themselves. +# +# Usage: +# monoprop_tier_targets(TIER OBJS_VAR VARIANT_INCLUDE_DIR_VAR ) +function(monoprop_tier_targets) + set( + _one_value_args + TIER + OBJS_VAR + VARIANT_INCLUDE_DIR_VAR + ) + cmake_parse_arguments(PARSE_ARGV 0 _arg "" "${_one_value_args}" "") + + if(_arg_TIER STREQUAL monoprop_FAT_BASELINE_TIER) + set(_objs "monoprop-objs") + else() + _monoprop_tier_slug("${_arg_TIER}" _slug) + set(_objs "monoprop-objs-${_slug}") + endif() + + if(_arg_OBJS_VAR) + set(${_arg_OBJS_VAR} "${_objs}" PARENT_SCOPE) + endif() + if(_arg_VARIANT_INCLUDE_DIR_VAR) + set( + ${_arg_VARIANT_INCLUDE_DIR_VAR} + "${PROJECT_BINARY_DIR}/variants/${_arg_TIER}/include" + PARENT_SCOPE + ) + endif() +endfunction() + +# Fan a source list out over every tier. A macro and not a function, so that relative source paths +# still resolve against the CMakeLists.txt that named them. +macro(monoprop_engine_sources) + foreach(_engine_target IN LISTS monoprop_ENGINE_OBJ_TARGETS) + target_sources(${_engine_target} PRIVATE ${ARGN}) + endforeach() +endmacro() diff --git a/cmake/compiler_flags/GNU.CXX.cmake b/cmake/compiler_flags/GNU.CXX.cmake index 7d67f56c..d30c5ba9 100644 --- a/cmake/compiler_flags/GNU.CXX.cmake +++ b/cmake/compiler_flags/GNU.CXX.cmake @@ -6,9 +6,17 @@ if(CMAKE_CXX_COMPILER_ID MATCHES GNU) ) endif() + # -ffp-contract=off, and why it is not a micro-optimization to be traded away: without it the + # compiler fuses a*b+c into an FMA wherever the target has one, which changes the rounding of the + # coefficient accumulation. Measured across ISA levels, every evolved term stays bit-identical and + # only the energy moves, by 1-2 ULP from -march=x86-64-v3 up. That is small and it is also exactly + # the wrong shape: with a fat binary the same wheel would answer differently depending on which CPU + # it landed on, and `just diff-baseline` could no longer be a byte-wise gate. The project has no + # -ffast-math and treats accumulation order as a contract, so contraction is off everywhere -- not + # only in the tiers -- to keep a source build, a wheel and every tier bit-comparable. set( monoprop_CXX_FLAGS - "-Wall -Wno-unknown-pragmas -Wno-sign-compare -Woverloaded-virtual -Wwrite-strings -Wextra -Wconversion -Wnon-virtual-dtor -Wcast-align -Wunused-parameter -fdiagnostics-color=always -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer" + "-Wall -Wno-unknown-pragmas -Wno-sign-compare -Woverloaded-virtual -Wwrite-strings -Wextra -Wconversion -Wnon-virtual-dtor -Wcast-align -Wunused-parameter -fdiagnostics-color=always -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -ffp-contract=off" ) set(CMAKE_CXX_FLAGS_RELEASE "-O3 -DNDEBUG") diff --git a/cpp/include/monoprop/Variants.h.in b/cpp/include/monoprop/Variants.h.in index 45ea84e2..0444c4c5 100644 --- a/cpp/include/monoprop/Variants.h.in +++ b/cpp/include/monoprop/Variants.h.in @@ -16,54 +16,25 @@ #include -/** - * @brief Declares a compile-time function that reports the active variant. - * - * Expands to a `consteval` function named `variant()` with a GNU - * `target("arch=...")` attribute bound to the provided architecture string. - * - * @param archstr Architecture suffix used in `arch=`. - */ -#define monoprop_VARIANT(archstr) \ - [[using gnu: target("arch=" archstr)]] consteval auto variant() noexcept -> std::string_view { \ - return "arch=" archstr; \ - } - -/** - * @brief Declares a compile-time function that reports the machine-dependent - * flags GCC applies for the given variant. - * - * Expands to a `consteval` function named `machine_flags()` with a GNU - * `target("arch=...")` attribute bound to the provided architecture string. The - * returned value is the cleaned, space-separated list of machine flags GCC uses - * for that architecture, as reported by `gcc -march= -Q --help=target`. - * - * @param archstr Architecture suffix used in `arch=`. - * @param flagsstr Machine-dependent flags reported for the architecture. - */ -#define monoprop_VARIANT_FLAGS(archstr, flagsstr) \ - [[using gnu: target("arch=" archstr)]] consteval auto variant_flags() noexcept -> std::string_view { \ - return flagsstr; \ - } +// Configured once per ISA variant by _monoprop_generate_variant_header (cmake/compiler_flags/ +// CXXFlags.cmake). A fat-binary build writes one copy per tier into a per-tier include directory +// which that tier's object library searches first, so these two answers are the tier's own -- they +// are how a loaded variant is identified from Python, which is the only way to tell whether the +// import-time dispatch picked what it was supposed to. +// +// The identity is per *translation unit*, not per function: monoprop tiers whole libraries rather +// than multiversioning individual functions, because the vectorization the tiers buy lands in headers +// that are inlined into their callers, and a per-function dispatch seam would suppress that inlining. +// This header therefore carries no target attributes; there is nothing to attach them to. namespace monoprop { -#if defined(__x86_64__) || defined(_M_X64) -[[using gnu: target("default")]] -#endif +/// Identifier of the ISA variant this translation unit was compiled for. consteval auto variant() noexcept -> std::string_view { - return "default"; + return "@monoprop_VARIANT_ID@"; } -#if defined(__x86_64__) || defined(_M_X64) -[[using gnu: target("default")]] -#endif +/// Machine-dependent flags the compiler resolved for this variant, space separated. consteval auto variant_flags() noexcept -> std::string_view { - return "@monoprop_DEFAULT_VARIANT_FLAGS@"; + return "@monoprop_VARIANT_MACHINE_FLAGS@"; } - -// clang-format off -@monoprop_VARIANTS@ - -@monoprop_VARIANT_FLAGS@ -// clang-format on } // namespace monoprop diff --git a/cpp/monoprop/CMakeLists.txt b/cpp/monoprop/CMakeLists.txt index 1b233211..bead901b 100644 --- a/cpp/monoprop/CMakeLists.txt +++ b/cpp/monoprop/CMakeLists.txt @@ -8,61 +8,96 @@ message( "Using hwloc: ${HWLOC_LINK_LIBRARIES} (version ${HWLOC_VERSION})" ) -target_sources( - monoprop-objs - PRIVATE - Evolution.cpp - MPFunctions.cpp - MPGraph.cpp - Validation.cpp -) - -target_compile_definitions( - monoprop-objs - PUBLIC - $<$:monoprop_ENABLE_MPI> - $<$:monoprop_WIDE_TERM_INDEX> +monoprop_engine_sources( + Evolution.cpp + MPFunctions.cpp + MPGraph.cpp + Validation.cpp ) -# flags to prepend -target_compile_options( - monoprop-objs - BEFORE - PUBLIC - "${monoprop_CXX_FLAGS}" +# Everything an engine object library needs beyond its sources. Factored out because the fat binary +# compiles the engine once per ISA tier and the tiers must differ in exactly two things -- the arch +# flags and which Variants.h they see -- with nothing else drifting apart between them. +# +# variant_include_dir goes in BEFORE the shared binary include dir, which is what makes a tier report +# its own identity instead of the build's default one. +function(_monoprop_configure_engine_objs target arch_flags variant_include_dir) + target_compile_definitions( + ${target} + PUBLIC + $<$:monoprop_ENABLE_MPI> + $<$:monoprop_WIDE_TERM_INDEX> + ) + + # flags to prepend + target_compile_options( + ${target} + BEFORE + PUBLIC + "${monoprop_CXX_FLAGS}" + "${arch_flags}" + ) + + # flags to append + target_compile_options(${target} PUBLIC "${EXTRA_CXXFLAGS}") + + target_include_directories( + ${target} + BEFORE + PUBLIC + $ + ) + + target_include_directories( + ${target} + PUBLIC + $ + $ + $ + ) + + target_link_libraries( + ${target} + PUBLIC + monoprop-sanitizers + Boost::boost + Threads::Threads + $<$:MPI::MPI_CXX> + PRIVATE + PkgConfig::HWLOC + ) + + set_target_properties( + ${target} + PROPERTIES + CXX_VISIBILITY_PRESET + hidden + VISIBILITY_INLINES_HIDDEN + YES + ) +endfunction() + +if(monoprop_ENABLE_FAT_BINARY) + foreach(_tier IN LISTS monoprop_FAT_TIERS) + monoprop_tier_targets( + TIER "${_tier}" + OBJS_VAR _tier_objs + VARIANT_INCLUDE_DIR_VAR _tier_variant_include_dir + ) + _monoprop_tier_spec(TIER "${_tier}" MARCH_VAR _tier_flags) + _monoprop_configure_engine_objs( + "${_tier_objs}" + "${_tier_flags}" + "${_tier_variant_include_dir}" + ) + endforeach() +else() + _monoprop_configure_engine_objs( + monoprop-objs "${ARCH_FLAG}" -) - -# flags to append -target_compile_options(monoprop-objs PUBLIC "${EXTRA_CXXFLAGS}") - -target_include_directories( - monoprop-objs - PUBLIC - $ - $ - $ -) - -target_link_libraries( - monoprop-objs - PUBLIC - monoprop-sanitizers - Boost::boost - Threads::Threads - $<$:MPI::MPI_CXX> - PRIVATE - PkgConfig::HWLOC -) - -set_target_properties( - monoprop-objs - PROPERTIES - CXX_VISIBILITY_PRESET - hidden - VISIBILITY_INLINES_HIDDEN - YES -) + "${PROJECT_BINARY_DIR}/include" + ) +endif() include(GenerateExportHeader) diff --git a/cpp/monoprop/detail/graph_encoding/CMakeLists.txt b/cpp/monoprop/detail/graph_encoding/CMakeLists.txt index ebf1faea..7193e2e4 100644 --- a/cpp/monoprop/detail/graph_encoding/CMakeLists.txt +++ b/cpp/monoprop/detail/graph_encoding/CMakeLists.txt @@ -8,4 +8,4 @@ target_sources( "MPGraphEncodingTypes.h" ) -target_sources(monoprop-objs PRIVATE MPGraphEncoding.cpp) +monoprop_engine_sources(MPGraphEncoding.cpp) diff --git a/cpp/monoprop/detail/mpi/CMakeLists.txt b/cpp/monoprop/detail/mpi/CMakeLists.txt index 5b9969b0..018f6154 100644 --- a/cpp/monoprop/detail/mpi/CMakeLists.txt +++ b/cpp/monoprop/detail/mpi/CMakeLists.txt @@ -15,4 +15,4 @@ target_sources( "ShmComm.h" ) -target_sources(monoprop-objs PRIVATE MPICompat.cpp) +monoprop_engine_sources(MPICompat.cpp) diff --git a/cpp/monoprop/detail/pare/CMakeLists.txt b/cpp/monoprop/detail/pare/CMakeLists.txt index a3233909..eb7e7f71 100644 --- a/cpp/monoprop/detail/pare/CMakeLists.txt +++ b/cpp/monoprop/detail/pare/CMakeLists.txt @@ -7,4 +7,4 @@ target_sources( "PareGraph.h" ) -target_sources(monoprop-objs PRIVATE PareGraph.cpp) +monoprop_engine_sources(PareGraph.cpp) diff --git a/cpp/monoprop/detail/partition/CMakeLists.txt b/cpp/monoprop/detail/partition/CMakeLists.txt index 19d4b69b..be02dd47 100644 --- a/cpp/monoprop/detail/partition/CMakeLists.txt +++ b/cpp/monoprop/detail/partition/CMakeLists.txt @@ -8,4 +8,4 @@ target_sources( "PartitionGroup.h" ) -target_sources(monoprop-objs PRIVATE CpuTopology.cpp) +monoprop_engine_sources(CpuTopology.cpp) diff --git a/docs/content/docs/building.mdx b/docs/content/docs/building.mdx index 87b80b83..323d4093 100644 --- a/docs/content/docs/building.mdx +++ b/docs/content/docs/building.mdx @@ -19,6 +19,22 @@ mechanism differs by build: The prebuilt wheels published to PyPI (`pip install monoprop`) are also built without MPI, so a from-source build is required for multi-rank runs. +### Other build-time options + +| Option | Default | Effect | +| --- | --- | --- | +| `monoprop_ENABLE_ARCH_FLAGS` | `ON` (`OFF` for the published wheels) | Compile with `-march=native` / `-xHost`. | +| `monoprop_ENABLE_FAT_BINARY` | `OFF` (`ON` for the published `x86-64` wheels) | Compile the engine once per x86-64 ISA tier and select one at import. See [The Fat Binary](/fat-binary). | +| `monoprop_FAT_MTUNE` | `skylake` | The `-mtune` value every fat-binary tier is scheduled for. Does **not** decide the AVX-512 vector width; the top tier ships at both. | +| `monoprop_WIDE_TERM_INDEX` | `OFF` | 64-bit term indices, for partitions holding more than ~2^32 terms. | +| `monoprop_ENABLE_MPI` | `OFF` | Multi-rank support, as above. | + +`monoprop_ENABLE_ARCH_FLAGS` and `monoprop_ENABLE_FAT_BINARY` are alternatives, not a +pair. A source build wants the first: `-march=native` uses every instruction the +machine has and beats any portable tier. A published wheel cannot use it at all, and +gets the second instead. With the fat binary on, the tiers supply the architecture +flags and `monoprop_ENABLE_ARCH_FLAGS` has no further effect. + ## Prerequisites - a C++23-compliant compiler; on Linux the minimum supported versions are **GCC 14** and **Clang 18** @@ -207,6 +223,8 @@ Python API under TSan would require a TSan-instrumented CPython. ### Related workflows +- Use `just build-fat` and `just test-variants` for the multi-ISA build described in + [The Fat Binary](/fat-binary). - Use `just test-wide` for the 64-bit `monoprop_WIDE_TERM_INDEX` configuration. - Use `just code-coverage` for the coverage build. - Use `ctest --test-dir build/editable/Release -L serial` or `-L mpi-2` to @@ -216,6 +234,7 @@ Python API under TSan would require a TSan-instrumented CPython. ## See also - [Getting Started](/getting-started) — installing a prebuilt release from PyPI. +- [The Fat Binary](/fat-binary) — how the published x86-64 wheels carry four ISA tiers, and how to pin one. - [Parallelism and distribution](/features/parallelism) — running across MPI ranks and shared-memory threads. - [Testing](/testing) — the full Python and C++ test workflow. - [How to Contribute](/how-to-contribute) — contributor workflow and documentation checks. diff --git a/docs/content/docs/fat-binary.mdx b/docs/content/docs/fat-binary.mdx new file mode 100644 index 00000000..c4fc7da7 --- /dev/null +++ b/docs/content/docs/fat-binary.mdx @@ -0,0 +1,349 @@ +--- +title: The Fat Binary +description: Why the published x86-64 wheels carry the engine five times over, how the right copy gets loaded, and how to pin one. +--- + +Every published `x86-64` wheel contains the propagation engine **five times**: once for each of four +instruction-set levels, and the top level twice at two vector widths. Importing `monoprop` picks the +one the CPU should be given. This page covers why, how, and what it costs. + +If you are on `aarch64` — Apple silicon, or an Arm server — none of this applies: those wheels carry +one copy, because there is one relevant ISA and nothing to choose between. + +## The problem it solves + +A source build compiles with `-march=native`, which lets the compiler use every instruction the build +machine has. That is the fastest thing available and it is also unshippable: the resulting binary +crashes with `SIGILL` on any older CPU. + +So the published wheels turned architecture flags **off** entirely. The consequence was easy to miss +and expensive: developers building from source got a fully vectorized library, while everyone +installing from PyPI got one compiled for the 2003 `x86-64` baseline. Not merely "less vectorized" — +at that baseline `std::popcount` has no instruction and compiles to `call __popcountdi2@PLT`, a +function call, in a library whose inner loops are made of population counts. + +A fat binary is the way out: ship several ISA levels and choose at run time. The alternative — one +wheel per microarchitecture, and users picking — moves the problem onto the user. + +## Why five tiers, and which five + +The tiers come from a compile-time study of what each ISA level actually buys this codebase, using +GCC's own `-fopt-info-vec-loop-all` reports across the psABI levels. Counting only project code, and +counting each vectorized site once per instantiation (a "raw" count — most of these loops live in +headers that are instantiated many times): + +| ISA level | loop-vectorized | SLP | total | +| --- | --- | --- | --- | +| `x86-64` (v1) | 37 | 204 | 241 | +| `x86-64-v2` | 67 | 204 | 271 | +| `x86-64-v3` | 98 | 224 | 322 | +| `x86-64-v4` | 124 | 228 | 352 | +| `x86-64-v4` + `avx512vpopcntdq` | 192 | 228 | 420 | + +Three things in that table decided the tier list. + +**SLP barely moves; loop vectorization is the whole story.** Basic-block vectorization goes 204 → 228 +across the entire range, because the fixed-trip word loops the scan is built from already vectorize +under plain SSE2. What the ISA buys is the *runtime*-trip loops: the wide-bitset paths, the +anticommutation fold, and the cutoff accumulation. + +**`v4` alone is not worth a tier; `v4` plus a vector popcount is.** Ablating the AVX-512 extensions one +at a time on top of `v4`, `-mavx512vpopcntdq` accounts for the entire `v4 → v4x` gain of +68 loops and +each of `avx512ifma`, `avx512vbmi`, `avx512vbmi2`, `avx512bitalg`, `gfni`, `vaes` and `vpclmulqdq` +accounts for exactly **zero**. That is this codebase's shape: `std::popcount` word-loop reductions from +top to bottom, and no intrinsics anywhere. So the top tier is `-march=x86-64-v4 -mavx512vpopcntdq`, and +plain `v4` is not shipped — it would be three quarters of a megabyte for nothing. + +**That top tier cannot be a psABI level, which rules out one implementation.** More on that below. + +**And the top tier ships twice.** Nothing about `-march` says how wide the vectors behind the AVX-512 +registers should be; `-mprefer-vector-width` does, and left unset GCC takes it from the `-mtune` +tables — so the width was being decided by a core chosen for its *schedule*. The two tiers pin it +instead, at 256 and 512, and the run time chooses. See +[Why the top tier ships at two vector widths](#why-the-top-tier-ships-at-two-vector-widths). + +The `v1` tier is shipped despite being the slowest, because it is the floor: it is what runs on a CPU +that predates SSE4.2, and without it such a machine has nothing to load. + +## Why `-mtune=skylake` everywhere + +`-march` and `-mtune` are separate decisions. `-march` sets which instructions may be used; `-mtune` +only changes the cost model and the instruction schedule, and never widens the ISA. So every tier can +tune for the same core without affecting which machines it runs on. + +The tuning choice turned out to matter *more* than any single ISA step. At a fixed +`-march=x86-64-v3`, sweeping ten `-mtune` values: + +| `-mtune` | distinct vectorized sites | loops rejected as `not profitable` | +| --- | --- | --- | +| `generic` | 89 | 849 | +| `skylake` | 126 | 148 | +| `haswell` | 128 | 148 | +| `sapphirerapids` | 129 | 148 | +| `znver3` | 127 | 148 | +| `znver4` / `znver5` | 119 | 237 | + +`-mtune=generic` deliberately optimizes for no particular core, and its cost model rejects loops at +roughly five times the rate of any concrete one. Almost any real core is better, and the differences +*between* real cores are small. + +There is no vendor-neutral option: GCC rejects `-mtune=x86-64-v3`. It is `generic` or a named core. +`skylake` is the choice because it sits in the top cluster on every counter, has the smallest `.text` +of those that get there, and is the oldest core in that cluster — so it is the least likely to schedule +badly on anything from 2015 onwards. The AMD penalty is negligible (`znver3` scores 127 against +`skylake`'s 126). Notably, tuning for the newest AMD core is a small *regression* here. + +Set `monoprop_FAT_MTUNE` at configure time to try another. + +## Why the top tier ships at two vector widths + +`x86-64-v4-vpopcntdq-vw256` and `x86-64-v4-vpopcntdq-vw512` are the same instruction set. Identical +`-march`, identical `__builtin_cpu_supports` requirements, identical everything a feature bit can +express. They differ in one flag: `-mprefer-vector-width`. + +The pair exists because that flag has a default and the default is wrong somewhere. Left unset, GCC +takes the AVX-512 vector width from the `-mtune` tables — so it was being decided by +`monoprop_FAT_MTUNE`, a core picked for its *schedule* on the reasoning that `-mtune` never changes +which instructions come out. For AVX-512 widths it does, and differently per core: `-mtune=skylake` +and `-mtune=znver4` resolve to 512, `-mtune=icelake-server` and `-mtune=sapphirerapids` to 256. A +build that fixes the width has guessed on the user's behalf. Carrying both and choosing at run time is +the only way the answer is right on more than one machine. + +**No feature bit answers this**, which is why the discriminator is a table of core names read through +`__builtin_cpu_is` rather than a `__builtin_cpu_supports` query. The question is not what the CPU can +execute — both tiers need exactly the same instructions — but how wide the datapath behind the +registers really is and what the core charges in clock frequency for lighting all of it up. CPUID +reports neither. + +A core is on the narrow list when it has either of the two reasons to keep 512-bit code out: + +- **a split datapath** — 512-bit operations run on 256-bit hardware, so the width halves the + instruction count and buys no throughput. AMD Zen 4 is the case, and the one entry that is measured + rather than reasoned: pinning the two tiers on an EPYC 9R14, the 256-bit tier wins the 127-qubit + kicked-Ising model by **1.1%** with the two three-sample ranges disjoint (358.9–361.3 ms against + 363.5–365.7) and ties on the 120-mode Hubbard one (2058.7 against 2063.2, spreads overlapping). GCC + tunes `znver4` the other way, so this is a correction to it, not an application of it. +- **a frequency penalty** — the core drops its clock while 512-bit code is in flight. Ice Lake + through Rocket Lake pay a measured ~175 MHz of peak, which is what put `-mprefer-vector-width=256` + in both compilers' Intel tuning to begin with. + +Not on the list: Zen 5, which has the full-width datapath Zen 4 lacks, and Sapphire Rapids onwards, +where the frequency penalty went away. Both are decisions against GCC's own `-mtune` tables, which +still say 256 for every Intel AVX-512 core; neither could be measured here, so both are the mechanism +argument rather than a number — and both are falsifiable with `monoprop_VARIANT`. + +Also not on the list, and not by choice: Skylake-SP, Cascade Lake and Cooper Lake, the parts the +256-bit default was introduced for. They are `x86-64-v4` with no vector popcount, so they never reach +this tier at all and their names would be dead entries. + +A core name GCC does not recognise is dropped with a warning rather than failing the build. That core +then gets the 512-bit tier, which costs a percent or so; the alternative is a build that fails on an +older toolchain over a tuning hint. + +Two consequences worth keeping in mind. Each tier now carries **two** predicates — `runnable` and +`preferred` — and conflating them makes the wide tier unpinnable on exactly the machines worth +comparing it on. And nothing but a disassembly tells the pair apart: every test, every number and +every symbol table agrees even if the flag stops arriving, which is why +`test_reported_machine_flags_widen_with_the_tier` asserts that the width *setting* differs while the +feature set does not. + +## Why whole libraries, not individual functions + +The obvious implementation is GCC's function multiversioning — `target_clones` on the hot kernels, with +an ifunc resolver picking a clone per call. The natural worry is that a dispatch boundary is an +inlining boundary, and the vectorization above lands overwhelmingly in *headers* inlined into their +callers. That worry is answerable: `__attribute__((flatten))` on the cloned function inlines the whole +call tree into each clone first, and it measurably works — clones built that way come out with `ymm` at +`arch=x86-64-v3` and `zmm` at `arch=x86-64-v4`, header templates and all. The location is favourable +too: 91% of the project's vectorized loops are in a single translation unit, and +`MonomialPropagator::build_evolve_result_` is a non-template function that is the sole caller of +`detail::build_layer`. + +It was rejected for two other reasons, both measured. + +**An attribute cannot widen code defined outside it.** The tier itself is expressible: in a plain +`target` attribute a comma separates *options*, so `target("arch=x86-64-v4,avx512vpopcntdq")` is valid +(only `target_clones`, where a comma separates *clones*, rejects it). What does not work is getting the +engine compiled under it. GCC's `ix86_can_inline_p` refuses to inline across an `arch` mismatch, so a +targeted wrapper around an untargeted body compiles to a four-instruction `jmp` — `flatten` does not +override that, and `always_inline` on the immediate callee buys exactly one level. Nor does +`#pragma GCC target`: a template defined outside the region and first instantiated inside it is emitted +at the command-line target, with zero `vpopcnt` or `zmm` in the object. + +So for header-resident code there is exactly one input that widens it: the translation unit's command +line. The pattern that gets there through a pragma is Highway's `foreach_target.h` — physically +re-include the kernels inside a per-tier namespace with every standard header hoisted above the region +— which would make the engine's `namespace monoprop` a macro-named inline namespace per tier, and would +emit the same four copies of the same code, built serially in one compiler process instead of four in +parallel. + +**And `flatten` does not survive this call tree.** Flattening inlines the layer engine's whole +instantiation fan-out into one function body, and `target_clones` then makes one copy of that body per +tier to optimize. Measured on the runtime-width engine, where the fan-out is a single translation unit: +compiling it went from **16.7 s** to *killed at 21 minutes having peaked around 100 GB of compiler +memory*. Not a slow build; an unbuildable one, against 16 GB CI runners. Collapsing the widest axis of +the fan-out did not rescue it either — still out of memory at **24.5 GB after 11 minutes**, against +**14.75 s in 740 MB** for the same code without clones. + +The cost is not the fan-out, it is that `flatten` concentrates it into one function where GCC's +per-function passes go superlinear, and then multiplies by the clone count. Which is the whole argument +for tiering by library: the emitted code is the same either way, but as separate translation units the +multiplication is linear and parallel rather than superlinear in one process. `target_clones` with +`flatten` is the right tool when a small self-contained kernel sits behind one call; here the kernel is +the entire scan. On this branch the fan-out is wider still — the engine is templated on the mode count, +so the scan is instantiated once per width across the generated binder translation units. + +The duplication *is* reducible in principle, by tiering only the translation unit the vectorized loops +land in and sharing a single baseline copy of the rest. That needs the propagation kernel to *have* one: +here it is header-resident and instantiated per mode width in the binding translation units, so the +narrowest seam available is the module boundary. Narrowing it is a follow-up to moving the engine +out of the headers, not something this build shape can do. + +So the unit of tiering is the whole engine. Each tier is a separate compile of every library +translation unit *plus* the binding translation unit — the latter matters, because +`MonomialPropagator`'s inline methods and every template it instantiates are compiled there too. + +The second obvious implementation is `glibc-hwcaps`: drop `libmonoprop.so` into +`glibc-hwcaps/x86-64-v{2,3,4}/` and let `ld.so` pick, with no code at all. That fails on the top tier. +The hwcaps directory names are the four psABI levels, and `x86-64-v4` **does not include** +`avx512vpopcntdq` — Skylake-X and Cascade Lake are `v4` and have no vector popcount. Installing the top +tier as `x86-64-v4` would hand it to those machines and they would fault. The selection predicate has +to be ours, so the dispatch has to be ours. + +## What actually happens on import + +The wheel looks like this: + +```text +monoprop/ +├── _isa.abi3.so # the CPU probe: baseline ISA, ~100 KB +├── _bootstrap.py # the selection +└── _variants/ + ├── x86-64-v1/_core.abi3.so + ├── x86-64-v2/_core.abi3.so + ├── x86-64-v3/_core.abi3.so + ├── x86-64-v4-vpopcntdq-vw256/_core.abi3.so + └── x86-64-v4-vpopcntdq-vw512/_core.abi3.so +``` + +There is no `monoprop/_core` of its own. `monoprop/_bootstrap.py` is imported first — its name sorts +ahead of `_core` so that alphabetical import ordering keeps it there — and it: + +1. asks `monoprop._isa` which variants this CPU should be *given*, best first; +2. takes the best one that is also installed; +3. loads it under the name `monoprop._core`. + +Every module in the package then imports `monoprop._core` as usual and is unaware that a choice was +made. Each variant is named `_core` on disk whatever tier it belongs to, because CPython derives the +initialization symbol it looks for from the last component of the module name being loaded. + +`_isa` exists as a separate, deliberately tiny extension because the question "what can this CPU do" +has to be answered *before* any tiered code is loaded. It is built for the baseline ISA and links no +engine code, so it is the one module guaranteed to load everywhere. It answers with +`__builtin_cpu_supports`, which consults `CPUID` **and** `XGETBV` — so an AVX-512-capable CPU under a +kernel or hypervisor that has not enabled the ZMM register state correctly reports the feature as +absent, which is what keeps the dispatch off machines that would fault. + +The probe answers two questions, not one, and the difference matters. `runnable_variants()` is the +capability answer — the CPU has the instructions — and is what a `monoprop_VARIANT` pin is checked +against. `supported_variants()` is what the CPU should be handed unasked, and is what the automatic +selection reads. They differ for exactly one variant, the 512-bit one: every AVX-512 CPU can run it +and only some should have it. Conflating the two would make it unpinnable on precisely the machines +worth comparing it on. + +One subtlety worth knowing if you work on the build: the baseline ISA is also applied globally, not +just to the baseline tier's objects. A wheel contains code from targets nobody tiered — nanobind's +static library, for one — and those compile with whatever `-march` the toolchain defaults to, which is +*not* necessarily the psABI baseline. GCC as shipped by recent Ubuntu is configured +`--with-arch-64=x86-64-v3`. Without a global floor, the `v1` and `v2` variants would carry AVX2 in +their glue code and `_isa` itself would fault on the machines it exists to detect. + +## The same answers on every tier + +The tiers change instruction selection. They must not change results, and that takes one deliberate +flag. + +Without it, `-march=x86-64-v3` and up fuse `a*b+c` into an FMA, which changes the rounding of the +coefficient accumulation. Measured across ISA levels: every evolved term stays bit-identical and only +the energy moves, by one or two units in the last place. Small — and exactly the wrong shape for a fat +binary, where it would mean the same wheel answering differently depending on which CPU it landed on. +So `-ffp-contract=off` is set project-wide, not only in the tiers, which keeps a source build, a wheel +and every tier bit-comparable. All four tiers produce byte-identical output. + +## Using it + +Which variant loaded: + +```python +import monoprop + +print(monoprop.__variant__) # e.g. "x86-64-v3", or "native" for a source build +print(monoprop.available_variants()) # every variant in this install, best first +print(monoprop.supported_variants()) # every variant this CPU could run, best first +``` + +`monoprop.__compiler_flags__["machine-flags"]` reports the flags the compiler resolved *for the loaded +variant*, so it is the authoritative answer to "what was this actually built with". + +To pin a variant — benchmarking a tier, reproducing a report, bisecting a codegen difference — set +`monoprop_VARIANT`: + +```bash +monoprop_VARIANT=x86-64-v2 python your_script.py +``` + +Naming a variant that is not installed, or one this CPU cannot execute, is an error rather than a +silent fallback: the point of pinning one is to know which one ran. A variant this CPU merely would +not have *chosen* is honoured, though — that is how the 512-bit tier gets measured on a Zen 4. + +## Building one + +The fat binary is **off** by default in source builds, because a source build has `-march=native`, +which beats every tier: + +```bash +just build-fat +``` + +or, directly: + +```bash +uv sync --all-extras -v --config-settings=cmake.define.monoprop_ENABLE_FAT_BINARY=ON +``` + +Note that a plain `uv run` afterwards re-syncs without that setting and silently replaces the fat +build with a single-ISA one; use `uv run --no-sync`. + +`just test-variants` runs the Python suite once per installed variant. It exists because the dispatch +always picks the *best* tier, which means the lower tiers would otherwise ship untested from every +developer machine and every CI runner. + +`monoprop_ENABLE_FAT_BINARY` is x86-64 only and needs GCC or Clang; requesting it elsewhere is a +configure error rather than a silently untiered build. The tier list and the flags live in +`cmake/compiler_flags/FatBinary.cmake`, which is the only place a tier is declared — the loader's +predicate table is generated from it, so a tier cannot be built without being selectable or selectable +without being built. + +## What it costs + +About **3.8 MB** of extension modules instead of 1 MB, plus the `~100 KB` probe. Four tiers are also +roughly four times the compile work of one, which lands on the `manylinux_x86_64` leg of the wheel +matrix. + +## What is still open + +- **None of the above is a measurement of speed.** It is a measurement of what the compiler emitted. + The tier list is chosen from codegen, and the timing round is what would settle whether `v2` earns + its place and whether the top tier's AVX-512 is a win in wall-clock as well as in instruction count. +- **`-mprefer-vector-width=256`** costs 25 vectorized loops statically, but AMD's Zen 4 implements + AVX-512 on a double-pumped 256-bit datapath while Intel's server parts do not. Which way that trade + goes cannot be counted, only timed, and it needs an Intel data point. +- **The `v1` floor's real audience.** Today's `ARCH_FLAGS=OFF` wheels get whatever the manylinux + image's GCC defaults to, which may already be above the baseline. If it is, the `v1` tier is + insurance rather than an improvement — worth knowing, since it is the one tier with a pathological + `std::popcount`. + +## See also + +- [Building from Source](/building) — the other build-time options, and the row-store crossover. +- [Benchmarks](/benchmarks) — the timing harness the open questions above need. diff --git a/docs/content/docs/meta.json b/docs/content/docs/meta.json index 66e879a2..00d22610 100644 --- a/docs/content/docs/meta.json +++ b/docs/content/docs/meta.json @@ -1,3 +1,3 @@ { - "pages": ["index", "getting-started", "building", "concepts", "features", "tutorials", "api", "benchmarks", "references", "how-to-contribute"] + "pages": ["index", "getting-started", "building", "fat-binary", "concepts", "features", "tutorials", "api", "benchmarks", "references", "how-to-contribute"] } diff --git a/justfile b/justfile index 9346525c..efa1b5fc 100644 --- a/justfile +++ b/justfile @@ -42,6 +42,28 @@ test-mpi RANKS='': ctest --test-dir build/editable/Release --output-on-failure; \ done +# Build the fat binary: the engine compiled once per x86-64 ISA tier, with one selected when +# monoprop is imported. This is what published wheels are, and it is *not* what a source build wants +# -- -march=native beats every tier -- so it is a separate recipe rather than the default. Note +# `uv run --no-sync` for everything afterwards: a plain `uv run` re-syncs without the config setting +# and silently replaces the fat build with a single-ISA one. + +build-fat: + uv sync --all-extras --group test --reinstall-package monoprop --no-cache -v --config-settings-package="monoprop:cmake.define.monoprop_ENABLE_FAT_BINARY=ON" + uv run --no-sync python -c 'import monoprop; print("loaded", monoprop.__variant__, "of", monoprop.available_variants())' + +# Run the Python suite once per installed ISA variant, not just the one this CPU selects. Without +# this the lower tiers ship untested on every developer machine and every CI runner, since the +# dispatch always picks the best one available. + +test-variants: + variants=$(uv run --no-sync python -c 'import monoprop; print(" ".join(monoprop.available_variants()))'); \ + if [ -z "$variants" ]; then echo "not a fat binary; run 'just build-fat' first" >&2; exit 1; fi; \ + for v in $variants; do \ + echo "=== monoprop_VARIANT=$v"; \ + monoprop_VARIANT="$v" uv run --no-sync python -m pytest -m "not mpi" -q; \ + done + # Build and run the C++ suite with a 64-bit TermIndex (monoprop_WIDE_TERM_INDEX=ON). # This is the only configuration that compiles the wide `#if defined(monoprop_WIDE_TERM_INDEX)` # branches (operator_index_tests, large_cosine_storage_tests, graph_encoding_tests), so it diff --git a/pyproject.toml b/pyproject.toml index f67f7d6d..0046a154 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -392,3 +392,15 @@ environment = { SKBUILD_CMAKE_ARGS = "-Dmonoprop_ENABLE_ARCH_FLAGS=OFF;-Dmonopro [tool.cibuildwheel.macos] before-all = "brew install boost hwloc" environment = { MACOSX_DEPLOYMENT_TARGET = "15.0", SKBUILD_CMAKE_ARGS = "-Dmonoprop_ENABLE_ARCH_FLAGS=OFF;-Dmonoprop_ENABLE_CXX_UNIT_TESTS=OFF;-Dmonoprop_ENABLE_MPI=OFF" } + +# The fat binary is x86-64 only, and x86-64 is only built on the linux-intel leg of the wheel matrix +# (linux-arm and macos-arm are both aarch64, where there is one ISA and nothing to select). So it goes +# on an override rather than in [tool.cibuildwheel.linux], which also covers manylinux_aarch64. +# +# ENABLE_ARCH_FLAGS stays OFF: it means -march=native, which for a published wheel means "whatever the +# build machine happened to be" -- the thing the tiers replace. Building four tiers is roughly four +# times the compile work of one, against build.tool-args = -j2; raise it here if this leg becomes the +# long pole. +[[tool.cibuildwheel.overrides]] +select = "*-manylinux_x86_64" +environment = { SKBUILD_CMAKE_ARGS = "-Dmonoprop_ENABLE_ARCH_FLAGS=OFF;-Dmonoprop_ENABLE_FAT_BINARY=ON;-Dmonoprop_ENABLE_CXX_UNIT_TESTS=OFF;-Dmonoprop_ENABLE_MPI=OFF" } diff --git a/src/monoprop/__init__.py b/src/monoprop/__init__.py index b36c791f..258efd07 100644 --- a/src/monoprop/__init__.py +++ b/src/monoprop/__init__.py @@ -18,6 +18,14 @@ import importlib.util +# Must precede ._core: on a fat-binary wheel there is no monoprop/_core to import until this module +# has bound one of the shipped ISA variants to that name. The module name sorts ahead of _core so +# alphabetical import ordering keeps it there. +from ._bootstrap import ( + available_variants, + runnable_variants, + supported_variants, +) from ._core import ( MAX_NUM_MODES, __build_type__, @@ -63,11 +71,14 @@ "__variant__", "__version__", "antihermitian_generator_correction", + "available_variants", "expand_monomials", "has_mpi", "integrals_to_fermion", "is_antihermitian", "jordan_wigner_basis_change", + "runnable_variants", + "supported_variants", "validate_parameter_mapping", ] diff --git a/src/monoprop/_bootstrap.py b/src/monoprop/_bootstrap.py new file mode 100644 index 00000000..f24b05f5 --- /dev/null +++ b/src/monoprop/_bootstrap.py @@ -0,0 +1,218 @@ +# Copyright 2026 Algorithmiq +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Binds ``monoprop._core`` to the ISA variant of the engine that this CPU can run. + +A wheel built as a fat binary carries the compiled engine once per x86-64 ISA tier, under +``monoprop/_variants//``, and no ``monoprop/_core`` of its own. Importing this module picks one +and registers it as ``monoprop._core``, so every other import in the package is unaffected. + +A single-ISA build -- any source build, and every non-x86-64 wheel -- ships ``monoprop/_core`` +directly. There is then nothing to choose and importing this module does nothing at all. + +This module is imported for its side effect, which is why the name sorts ahead of ``_core``: the +selection has to be in place before the first ``from ._core import ...`` runs, and import sorters +order the package's imports alphabetically. +""" + +from __future__ import annotations + +import importlib.machinery +import importlib.util +import os +import sys +from pathlib import Path +from typing import TYPE_CHECKING + +if TYPE_CHECKING: + from types import ModuleType + +_CORE_MODULE = "monoprop._core" +_VARIANTS_DIRNAME = "_variants" + +#: Pin the variant instead of probing the CPU. Refused if the named variant is absent or unrunnable, +#: because the point of pinning one is to know which one ran -- benchmarking a tier, reproducing a +#: report, bisecting a codegen difference. +VARIANT_ENV_VAR = "monoprop_VARIANT" + + +def _variants_root() -> Path | None: + """The ``_variants`` directory, or ``None`` if this build has none. + + Searched along the package's ``__path__`` rather than next to this file, because an editable + install has two entries there: the source tree, holding the Python modules, and the CMake install + tree, holding everything compiled. ``_variants`` is only ever in the second. + """ + package = sys.modules.get(__package__ or "monoprop") + search = list( + getattr(package, "__path__", None) or [str(Path(__file__).resolve().parent)] + ) + for entry in search: + candidate = Path(entry) / _VARIANTS_DIRNAME + if candidate.is_dir(): + return candidate + return None + + +def _module_path(directory: Path) -> Path | None: + """The compiled core inside one variant directory, or ``None`` if there is none.""" + # Both spellings occur: a stable-ABI build writes _core.abi3.so, a version-specific one writes + # _core.cpython--.so. EXTENSION_SUFFIXES holds whichever this interpreter accepts. + for suffix in importlib.machinery.EXTENSION_SUFFIXES: + candidate = directory / f"_core{suffix}" + if candidate.is_file(): + return candidate + return None + + +def _probe() -> ModuleType | None: + try: + # Deliberately not a top-level import: on a single-ISA build the probe is not shipped at all, + # and importing this module must still work there. + from . import _isa # noqa: PLC0415 + except ImportError: + return None + return _isa + + +def available_variants() -> tuple[str, ...]: + """ISA variants installed alongside this package, best first. + + Empty for a single-ISA build, which is the shape of every source build. + """ + root = _variants_root() + if root is None: + return () + on_disk = { + entry.name + for entry in root.iterdir() + if entry.is_dir() and _module_path(entry) is not None + } + probe = _probe() + known = tuple(probe.known_variants()) if probe is not None else () + ordered = [name for name in known if name in on_disk] + # Anything on disk the probe does not know about is still reported, so a mismatch between the two + # shows up as an odd listing rather than as a variant that silently never gets picked. + ordered.extend(sorted(on_disk.difference(known))) + return tuple(ordered) + + +def supported_variants() -> tuple[str, ...]: + """ISA variants this CPU should be given, best first, whether installed or not. + + Not the same as the ones it *can* execute -- see :func:`runnable_variants`. Two variants differ + only in vector width, which is a tuning question and not a capability one, so a CPU that can run + 512-bit code but is measurably better off without it does not offer that variant here. + """ + probe = _probe() + return tuple(probe.supported_variants()) if probe is not None else () + + +def runnable_variants() -> tuple[str, ...]: + """ISA variants the running CPU has the instructions for, best first. + + A superset of :func:`supported_variants`, and the one a ``monoprop_VARIANT`` pin is checked + against: pinning a variant this CPU merely would not have chosen is the whole point of pinning, + while pinning one it cannot execute is a SIGILL somewhere inside the scan. + """ + probe = _probe() + return tuple(probe.runnable_variants()) if probe is not None else () + + +def _select(available: tuple[str, ...]) -> str: + supported = supported_variants() + requested = os.environ.get(VARIANT_ENV_VAR) + + if requested: + if requested not in available: + raise RuntimeError( + f"{VARIANT_ENV_VAR}={requested!r} is not installed; " + f"available variants: {', '.join(available)}" + ) + # Runnable, not supported: a pin this CPU merely would not have chosen is honoured, since + # comparing it against the one that would have been is the reason for pinning. + runnable = runnable_variants() + if runnable and requested not in runnable: + raise RuntimeError( + f"{VARIANT_ENV_VAR}={requested!r} needs instructions this CPU does not have; " + f"runnable variants: {', '.join(runnable)}" + ) + return requested + + if not supported: + raise ImportError( + "monoprop was built as a fat binary but its CPU probe (monoprop._isa) is missing or " + "unusable, so no ISA variant can be selected. This is a broken installation; " + f"reinstall, or pin a variant with {VARIANT_ENV_VAR}." + ) + + for variant in supported: + if variant in available: + return variant + + raise ImportError( + f"none of the installed ISA variants ({', '.join(available)}) can run on this CPU, " + f"which supports {', '.join(supported)}" + ) + + +def _load(variant: str) -> ModuleType: + root = _variants_root() + path = None if root is None else _module_path(root / variant) + if ( + path is None + ): # pragma: no cover - _select only returns variants that have a module + raise ImportError(f"ISA variant {variant!r} has no compiled core") + + spec = importlib.util.spec_from_file_location(_CORE_MODULE, path) + if spec is None or spec.loader is None: # pragma: no cover - defensive + raise ImportError(f"cannot load ISA variant {variant!r} from {path}") + + module = importlib.util.module_from_spec(spec) + # Registered before execution, as extension modules expect, and removed again on failure so a + # retry does not find a half-initialized module. + sys.modules[_CORE_MODULE] = module + try: + spec.loader.exec_module(module) + except BaseException: + del sys.modules[_CORE_MODULE] + raise + return module + + +def install_core() -> str: + """Register the selected variant as ``monoprop._core``; return its id, or ``""``. + + Returns the empty string for a single-ISA build, where the normal import machinery finds + ``monoprop._core`` on its own and there is nothing to select. + """ + if _CORE_MODULE in sys.modules: + return str(getattr(sys.modules[_CORE_MODULE], "__variant__", "")) + + available = available_variants() + if not available: + return "" + + variant = _select(available) + module = _load(variant) + # Also as an attribute of the parent package, which is what the import system would have done and + # what `monoprop._core` attribute access relies on. + parent = sys.modules.get("monoprop") + if parent is not None: + parent._core = module # type: ignore[attr-defined] + return variant + + +selected_variant = install_core() +"""ISA variant bound to ``monoprop._core``, or ``""`` for a single-ISA build.""" diff --git a/src/monoprop/bindings/CMakeLists.txt b/src/monoprop/bindings/CMakeLists.txt index 9994563d..45619fd4 100644 --- a/src/monoprop/bindings/CMakeLists.txt +++ b/src/monoprop/bindings/CMakeLists.txt @@ -103,32 +103,6 @@ configure_file( @ONLY ) -nanobind_add_module(_core - NB_SUPPRESS_WARNINGS # suppress warnings from nanobind and Python headers - NOMINSIZE # don’t perform optimizations to minimize binary size - LTO # perform link-time optimization - BACKEND_MODULE nanobind_backend - ${CMAKE_CURRENT_BINARY_DIR}/bindings.cpp - ${_bind_cpps} # List of generated binders -) - -target_include_directories( - _core - SYSTEM - PRIVATE - $,${mpi4py_INCLUDE_DIRS},> - PRIVATE - ${CMAKE_CURRENT_BINARY_DIR} - ${CMAKE_CURRENT_SOURCE_DIR} -) - -target_link_libraries( - _core - PRIVATE - monoprop-objs - monoprop-sanitizers -) - if(monoprop_SANITIZER STREQUAL "asan-ubsan") # GCC bounds-strict misdiagnoses nanobind's valid access to a tuple's trailing allocation. if(CMAKE_CXX_COMPILER_ID MATCHES GNU) @@ -153,39 +127,152 @@ COMMAND_ERROR_IS_FATAL ANY ) if(APPLE) - set(_rpath "@loader_path/${CMAKE_INSTALL_LIBDIR}") + set(_rpath_origin "@loader_path") else() - set(_rpath "\$ORIGIN/${CMAKE_INSTALL_LIBDIR}") + set(_rpath_origin "$ORIGIN") endif() -set_target_properties( - _core - PROPERTIES - MACOSX_RPATH - ON - SKIP_BUILD_RPATH - OFF - BUILD_WITH_INSTALL_RPATH - OFF +# Build one binding module. +# +# The module's sources are the configured bindings.cpp plus the generated per-mode-width batches, and +# every tier compiles all of them: `objs` is the engine object library this module links, and an engine +# object library carries its arch flags as PUBLIC compile options, so these translation units are +# compiled at the tier rather than merely linked against it. That is what makes the tier worth +# anything here -- MonomialPropagator is header-resident, so the scan, the fold and every +# template they instantiate land in these TUs and nowhere else. +# +# Every module is named `_core` on disk whatever its target is called, because CPython derives the +# init function it looks for from the last component of the module name it is asked to load, and the +# loader asks for `monoprop._core` for all of them. `destination` and `output_dir` are what keep the +# identically-named files apart. +function(_monoprop_add_binding_module) + set( + _one_value_args + TARGET + OBJS + DESTINATION + OUTPUT_DIR INSTALL_RPATH - "${_rpath}" - INSTALL_RPATH_USE_LINK_PATH - ON -) + ) + cmake_parse_arguments(PARSE_ARGV 0 _arg "" "${_one_value_args}" "") + + nanobind_add_module(${_arg_TARGET} + NB_SUPPRESS_WARNINGS # suppress warnings from nanobind and Python headers + NOMINSIZE # don’t perform optimizations to minimize binary size + LTO # perform link-time optimization + BACKEND_MODULE nanobind_backend # nanobind 3 split mode; rules out STABLE_ABI and NB_STATIC + ${CMAKE_CURRENT_BINARY_DIR}/bindings.cpp + ${_bind_cpps} # List of generated binders + ) -install(TARGETS _core LIBRARY DESTINATION ${PROJECT_NAME}) + target_include_directories( + ${_arg_TARGET} + SYSTEM + PRIVATE + $,${mpi4py_INCLUDE_DIRS},> + PRIVATE + ${CMAKE_CURRENT_BINARY_DIR} + ${CMAKE_CURRENT_SOURCE_DIR} + ) + + target_link_libraries( + ${_arg_TARGET} + PRIVATE + ${_arg_OBJS} + monoprop-sanitizers + ) + + set_target_properties( + ${_arg_TARGET} + PROPERTIES + OUTPUT_NAME + "_core" + LIBRARY_OUTPUT_DIRECTORY + "${_arg_OUTPUT_DIR}" + MACOSX_RPATH + ON + SKIP_BUILD_RPATH + OFF + BUILD_WITH_INSTALL_RPATH + OFF + INSTALL_RPATH + "${_arg_INSTALL_RPATH}" + INSTALL_RPATH_USE_LINK_PATH + ON + ) + + install(TARGETS ${_arg_TARGET} LIBRARY DESTINATION ${_arg_DESTINATION}) +endfunction() + +if(monoprop_ENABLE_FAT_BINARY) + foreach(_tier IN LISTS monoprop_FAT_TIERS) + monoprop_tier_targets(TIER "${_tier}" OBJS_VAR _tier_objs) + _monoprop_tier_slug("${_tier}" _slug) + _monoprop_add_binding_module( + TARGET _core-${_slug} + OBJS ${_tier_objs} + DESTINATION ${PROJECT_NAME}/_variants/${_tier} + OUTPUT_DIR ${CMAKE_CURRENT_BINARY_DIR}/_variants/${_tier} + # two levels deeper than a single-variant build: monoprop/_variants// -> monoprop/lib + INSTALL_RPATH "${_rpath_origin}/../../${CMAKE_INSTALL_LIBDIR}" + ) + endforeach() + + # The probe the import-time dispatch runs before it can load anything else, so it is the one + # extension that must execute on every machine the wheel installs on: baseline ISA, appended last so + # a CXXFLAGS from the environment cannot widen it, and linked against no engine object library so it + # cannot inherit a tier's flags either. + # + # Split mode like every other module here, and not by preference: wheel.py-api = "cp311" sets + # SKBUILD_SABI_VERSION, and nanobind 3 refuses a linked (NB_STATIC) module under any py-api below + # cp312. It costs the probe nothing -- the shared backend is not an engine object library, so the + # probe still inherits no tier's arch flags, and the fat binary's baseline floor in CMAKE_CXX_FLAGS + # is what keeps the backend itself at x86-64. + nanobind_add_module(_isa + NB_SUPPRESS_WARNINGS + NOMINSIZE + BACKEND_MODULE nanobind_backend + ${CMAKE_CURRENT_SOURCE_DIR}/isa.cpp + ) + target_include_directories(_isa PRIVATE ${PROJECT_BINARY_DIR}/include) + target_compile_options( + _isa + PRIVATE + -march=x86-64 + -mtune=${monoprop_FAT_MTUNE} + ) + install(TARGETS _isa LIBRARY DESTINATION ${PROJECT_NAME}) +else() + _monoprop_add_binding_module( + TARGET _core + OBJS monoprop-objs + DESTINATION ${PROJECT_NAME} + OUTPUT_DIR ${CMAKE_CURRENT_BINARY_DIR} + INSTALL_RPATH "${_rpath_origin}/${CMAKE_INSTALL_LIBDIR}" + ) +endif() + +# Which module the typing stub is generated from. Every tier exposes the same API -- they differ only +# in the instructions behind it -- so the baseline is as good as any, and it is the one guaranteed to +# import on the machine doing the build. +if(monoprop_ENABLE_FAT_BINARY) + list(GET monoprop_FAT_TIERS 0 _stub_tier) + set(_stub_python_path "${PROJECT_NAME}/_variants/${_stub_tier}") +else() + set(_stub_python_path "${PROJECT_NAME}") +endif() # generation of Python typing stubs -# NOTE must come after installing the _core target +# NOTE must come after installing the modules above # # Skip sanitizer builds: INSTALL_TIME stub generation imports _core and requires a preloaded # sanitizer runtime throughout installation. if(monoprop_SANITIZER STREQUAL "none") - nanobind_add_stub(_core + nanobind_add_stub(_core_stub INSTALL_TIME # Stub generation postponed to the installation phase. MODULE _core # Specifies the name of the module that should be imported. OUTPUT "${PROJECT_NAME}/_core.pyi" # Specifies the name of the stub file that should be written. - PYTHON_PATH "${PROJECT_NAME}" # List of search paths - relative to CMAKE_INSTALL_PREFIX - that should be considered when importing the module. + PYTHON_PATH "${_stub_python_path}" # List of search paths - relative to CMAKE_INSTALL_PREFIX - that should be considered when importing the module. VERBOSE # Show status messages generated by stubgen. MARKER_FILE "${PROJECT_NAME}/py.typed" # Automatically generate an empty marker file. ) diff --git a/src/monoprop/bindings/isa.cpp b/src/monoprop/bindings/isa.cpp new file mode 100644 index 00000000..b3815c4d --- /dev/null +++ b/src/monoprop/bindings/isa.cpp @@ -0,0 +1,97 @@ +// Copyright 2026 Algorithmiq +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +// The fat binary's CPU probe: the smallest possible extension module, built for the baseline ISA and +// linked against no engine object library, whose only job is to answer "which of the shipped variants +// can this machine execute". It exists because that question has to be answered *before* any tiered +// code is loaded, and answering it in Python would mean parsing /proc/cpuinfo -- Linux-only, and +// wrong about whether the OS has actually enabled the AVX-512 register state. +// +// __builtin_cpu_supports gets that right: it checks CPUID and XGETBV, so an AVX-512-capable CPU under +// a kernel or hypervisor that has not enabled ZMM state reports the feature as absent, which is the +// answer that keeps us from taking SIGILL. + +#include +#include + +#include +#include +#include + +#include "monoprop/FatVariants.h" + +namespace nb = nanobind; + +namespace { +/// One shipped variant: its id, whether this CPU can execute it, and whether it should be given it. +struct VariantRow final { + std::string id; + bool runnable; ///< the CPU has the instructions: what a monoprop_VARIANT pin is allowed to ask for + bool preferred; ///< the CPU should be handed this one unasked: what the automatic selection reads +}; + +// Every shipped variant, best first. Both answers come off the same generated table so they cannot +// drift apart, and they differ for exactly one variant -- see prefers_narrow_vectors() there. +auto variant_rows() -> std::vector { + auto out = std::vector{}; +#if defined(__x86_64__) || defined(_M_X64) + // Required before any other __builtin_cpu_* call in a translation unit that may run before + // libgcc's own constructor has. + __builtin_cpu_init(); +#define monoprop_ADD_VARIANT(id, runnable, preferred) \ + out.emplace_back(id, static_cast(runnable), static_cast(preferred)); + monoprop_FAT_VARIANT_TABLE(monoprop_ADD_VARIANT) +#undef monoprop_ADD_VARIANT +#endif + return out; +} + +auto filtered(bool VariantRow::*field) -> std::vector { + auto out = std::vector{}; + for (const auto &row : variant_rows()) { + if (row.*field) { + out.push_back(row.id); + } + } + return out; +} + +auto supported_variants() -> std::vector { + return filtered(&VariantRow::preferred); +} + +auto runnable_variants() -> std::vector { + return filtered(&VariantRow::runnable); +} + +auto known_variants() -> std::vector { + auto out = std::vector{}; + for (const auto &row : variant_rows()) { + out.push_back(row.id); + } + return out; +} +} // namespace + +NB_MODULE(_isa, mod) { + mod.doc() = "CPU feature probe for the fat binary's import-time variant selection."; + + mod.def("supported_variants", + &supported_variants, + "ISA variants this CPU should be given, best first -- what the automatic selection reads."); + mod.def("runnable_variants", + &runnable_variants, + "ISA variants this CPU can execute, best first -- a superset of supported_variants()."); + mod.def("known_variants", &known_variants, "ISA variants this build ships, best first, regardless of CPU support."); +} diff --git a/tests/test_variants.py b/tests/test_variants.py new file mode 100644 index 00000000..7d9a03ee --- /dev/null +++ b/tests/test_variants.py @@ -0,0 +1,205 @@ +# Copyright 2026 Algorithmiq +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""The fat binary: which ISA variant gets loaded, and whether they all agree. + +Skipped wholesale on a single-ISA build (any source build without +``-Dmonoprop_ENABLE_FAT_BINARY=ON``), where there is nothing to select. + +Every variant has to be exercised in a subprocess: the selection happens once, when ``monoprop`` is +first imported, and cannot be redone in-process. +""" + +from __future__ import annotations + +import itertools +import json +import os +import subprocess +import sys + +import pytest + +import monoprop +from monoprop._bootstrap import VARIANT_ENV_VAR + +INSTALLED = monoprop.available_variants() + +pytestmark = pytest.mark.skipif( + not INSTALLED, reason="not a fat binary: only one ISA variant is installed" +) + +# The variants that are one instruction set at two vector widths, told apart by this suffix on the id. +_WIDTH_SUFFIX = "-vw" + +# One rotation on a weight-2 observable inside a wider register, evaluated as bits rather than as a +# float, so "the tiers agree" means bit-for-bit and not "to some tolerance". The point is not the +# value: it is that -ffp-contract=off holds the value fixed across ISA levels that would otherwise +# contract a*b+c differently. +_PROBE = """ +import json, monoprop +from monoprop import Circuit, ExpGate, MajoranaOperator, MajoranaPropagator + +num_modes = 40 +observable = MajoranaOperator({(0, 1): 1j, (2, 3): 0.5j, (0, 1, 2, 3): 0.25}, num_modes) +gates = [ + ExpGate(MajoranaOperator({(2, 3): 1j}, num_modes)), + ExpGate(MajoranaOperator({(1, 2): 1j}, num_modes)), + ExpGate(MajoranaOperator({(0, 3): 1j}, num_modes)), +] +circuit = Circuit(gates=gates, system_size=num_modes, initial_state=[0, 1]) + +mp = MajoranaPropagator(observable, [0, 1], cutoff=8) +mp.build_graph(circuit) +angles = [0.37, -1.21, 0.05] +energy, gradient = mp.expectation_value_and_gradient(angles) + +print(json.dumps({ + "variant": monoprop.__variant__, + "machine_flags": monoprop.__compiler_flags__["machine-flags"], + "energy": float(energy).hex(), + "gradient": [float(g).hex() for g in gradient], +})) +""" + + +def _run_variant(variant: str) -> dict: + env = dict(os.environ, **{VARIANT_ENV_VAR: variant}) + completed = subprocess.run( # noqa: S603 - the interpreter running this test, not user input + [sys.executable, "-c", _PROBE], + env=env, + capture_output=True, + text=True, + check=True, + ) + return json.loads(completed.stdout) + + +@pytest.fixture(scope="module") +def per_variant() -> dict[str, dict]: + return {variant: _run_variant(variant) for variant in INSTALLED} + + +def test_the_loaded_variant_is_one_this_build_ships(): + assert monoprop.__variant__ in INSTALLED + + +@pytest.mark.skipif( + os.environ.get(VARIANT_ENV_VAR) is not None, + reason=f"{VARIANT_ENV_VAR} pins the variant, so there is no automatic choice to check", +) +def test_an_unpinned_run_loads_a_variant_this_cpu_is_offered(): + # supported_variants() is what the selection chooses from, which is *not* the same as what this CPU + # can execute: a pinned run can legitimately be on a variant absent from it, which is what makes the + # 512-bit tier measurable on a machine that would not have picked it. + assert monoprop.__variant__ in monoprop.supported_variants() + + +@pytest.mark.skipif( + os.environ.get(VARIANT_ENV_VAR) is not None, + reason=f"{VARIANT_ENV_VAR} pins the variant, so there is no automatic choice to check", +) +def test_selection_takes_the_best_supported_variant(): + # supported_variants() is ordered best first, so the first entry that is also installed is the + # only correct answer -- anything else means the dispatch is leaving performance on the table. + best = next(v for v in monoprop.supported_variants() if v in INSTALLED) + assert monoprop.__variant__ == best + + +def test_the_probe_and_the_install_agree_on_the_tier_list(): + # A tier known to the probe but never installed is silently unreachable; one installed but unknown + # to the probe can never be selected. Either way the wheel quietly loses a tier. + # not a top-level import: absent on a single-ISA build + from monoprop import _isa # noqa: PLC0415 + + assert set(_isa.known_variants()) == set(INSTALLED) + + +def test_every_variant_reports_its_own_identity(per_variant): + # Guards the provenance: without a per-tier Variants.h every variant would claim the build's + # default ISA, and there would be no way to tell which one had actually loaded. + for variant, result in per_variant.items(): + assert result["variant"] == variant + + +def test_every_variant_returns_bit_identical_numbers(per_variant): + reference_name, reference = next(iter(per_variant.items())) + for variant, result in per_variant.items(): + assert result["energy"] == reference["energy"], ( + f"{variant} disagrees with {reference_name} on the energy" + ) + assert result["gradient"] == reference["gradient"], ( + f"{variant} disagrees with {reference_name} on the gradient" + ) + + +def test_the_narrow_vector_variant_is_offered_wherever_the_wide_one_is(): + # The 256-bit variant asks for strictly less than the 512-bit one -- the same instructions, minus + # the claim that this core is worth handing zmm to -- so it is the fallback, and a CPU offered the + # wide variant and not the narrow one would mean the pair had been ordered wrong in + # monoprop_FAT_TIERS. + supported = monoprop.supported_variants() + for variant in [v for v in supported if v.endswith(f"{_WIDTH_SUFFIX}512")]: + narrow = variant.removesuffix("512") + "256" + assert narrow in supported, f"{variant} is supported here but {narrow} is not" + + +def test_a_variant_this_cpu_would_not_choose_is_still_pinnable(): + # The runnable/supported split exists for exactly this: the two vector-width variants require + # identical instructions, so both are always runnable, and refusing to pin the one the tuning table + # steers away from would make it unmeasurable on the machines worth measuring it on. + runnable = set(monoprop.runnable_variants()) + for variant in INSTALLED: + if variant.endswith((f"{_WIDTH_SUFFIX}512", f"{_WIDTH_SUFFIX}256")): + assert variant in runnable + + +def test_reported_machine_flags_widen_with_the_tier(per_variant): + def tokens(variant: str, *, settings: bool) -> set[str]: + return { + token + for token in per_variant[variant]["machine_flags"].split() + # "-mfoo=value" is a setting; a bare "-mfoo" is a feature and "-mno-foo" its negation + if ("=" in token) == settings and not token.startswith("-mno-") + } + + # INSTALLED is best first, so walking it backwards walks the tiers upwards. + for lower, higher in itertools.pairwise(reversed(INSTALLED)): + if lower.split(_WIDTH_SUFFIX)[0] == higher.split(_WIDTH_SUFFIX)[0]: + # The width pair, which is the one step up the ladder that adds no instruction: it must + # widen nothing and change the width setting, or the two tiers are the same build twice. + assert tokens(lower, settings=False) == tokens(higher, settings=False) + assert tokens(lower, settings=True) != tokens(higher, settings=True) + else: + assert tokens(lower, settings=False) < tokens(higher, settings=False), ( + f"{higher} does not strictly widen {lower}" + ) + + +def test_pinning_an_uninstalled_variant_is_refused(): + env = dict(os.environ, **{VARIANT_ENV_VAR: "x86-64-v99"}) + completed = subprocess.run( + [sys.executable, "-c", "import monoprop"], + env=env, + capture_output=True, + text=True, + check=False, + ) + assert completed.returncode != 0 + assert "is not installed" in completed.stderr + + +@pytest.mark.parametrize("variant", INSTALLED) +def test_pinning_an_installed_variant_selects_exactly_it(variant): + assert _run_variant(variant)["variant"] == variant