Skip to content

publish/build.py path has no preflight for the pybind interpreter mismatch that vs2019 already guards #121

Description

@wdolinar

Summary

vs2019_build.check_python_versions is a purpose-built preflight for the pybind
interpreter/target mismatch, and its failure message is excellent — it names the
cause and both remedies. But it is wired only into the xmsconan vs2019 command.
The everyday path (xmsconan publish → generated build.pyXmsConanPackager.run)
has no equivalent, so the identical mismatch surfaces as a raw CMake error deep
inside conan create, after the non-pybind configurations have already built.

Repro

xmsvtk @ 572697f, macOS arm64, xmsconan 2.24.0, repo .venv on CPython 3.14.2:

$ xmsconan publish
...
 >> The following configurations failed to build:
|  3  |  gnu17  |  |  Release  |  apple-clang  |  17  | armv8 | builtin | True | False | 3.13 |
errors when building... exiting...

The actual error is only visible in the Conan build folder's CMakeCache.txt
(Python3_DIR:PATH=Python3_DIR-NOTFOUND); reproduced standalone it is:

CMake Error: Could NOT find Python3: Found unsuitable version "3.14.2",
but required is exact version "3.13"
(found /Users/bill/dev/xms/xmsvtk/.venv/bin/python3,
 found components: Interpreter Development.Module)

PYTHON_TARGET_VERSION=3.14 xmsconan publish succeeds and produces a correct
cp314 wheel, confirming nothing is wrong with the library itself.

Why

Two sources disagree about which Python the build targets:

  • xmsconan/generator_tools/templates/CMakeLists.txt.jinja:24
    find_package(Python3 ${PYTHON_TARGET_VERSION} EXACT REQUIRED COMPONENTS Interpreter Development.Module),
    where PYTHON_TARGET_VERSION comes from the Conan option python_version.
  • xmsconan/xms_conan2_file.py:509_get_python_cmake_hints() pins
    Python3_EXECUTABLE to sys.executable, i.e. whatever interpreter runs Conan.

The generated build.py never passes python_versions to XmsConanPackager,
so the target resolves through [matrix]$PYTHON_TARGET_VERSION
DEFAULT_PYTHON_VERSIONS = ["3.13"] (packager.py:272). Any workstation whose
venv is not 3.13 therefore fails every pybind configuration.

The docstrings at vs2019_build.py:32-43 and 767-784 already describe this
precisely — the analysis is correct, it just is not applied to the publish path.

Why the existing lever doesn't help

There is no build.toml key that can pin this:

  • _MATRIX_VALUES (packager.py:285) accepts only compiler_runtime and
    pybind_build_types.
  • [ci].{,mac_,linux_}python_versions feeds only ci_file_generator.py, not the
    build matrix.

So PYTHON_TARGET_VERSION in the environment is the only control, and it is
undiscoverable from the failure. CI is unaffected because generated workflows set
it explicitly — which is exactly why this only ever bites on a workstation.

Worth noting: this became much easier to hit once a repo moves to a Python the
default does not name. xmsvtk moved to 3.14 ([ci] linux_python_versions = ["3.14"]),
its venv was rebuilt on 3.14, CI stayed green, and only the local publish broke.

Suggested fixes

  1. Run the preflight on the publish path (smallest, highest value). Reuse
    check_python_versions, or the same check against the generated+filtered
    matrix, in ci_tools/publish.py / XmsConanPackager.run before the first
    conan create. Today configs 1–2 build, 3 fails, and 4–5 build anyway, so the
    developer waits out the entire run to learn about a mismatch knowable up front.

  2. Decouple the interpreters (real fix). Have _get_python_cmake_hints()
    resolve an interpreter matching self.options.python_version rather than
    unconditionally using sys.executable. In the repro above CPython 3.13.9 was
    installed at /Library/Frameworks/Python.framework/Versions/3.13, so the 3.13
    build could have simply succeeded. "The Python running Conan" and "the Python we
    are building bindings for" are only the same by coincidence.

  3. Give build.toml a lever — add python_versions to _MATRIX_VALUES so a
    repo can pin its target next to [ci] instead of relying on an env var no
    generated file mentions.

(1) alone converts an hour-long confusing failure into an immediate, actionable
message. (2) removes the class of bug.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions