Enable Axom's Python to be installed via uv/pip by kennyweiss · Pull Request #1936 · llnl/axom

kennyweiss · 2026-08-01T00:30:38Z

Summary

This feature PR packages Axom's Python bindings as a wheel that can be installed via uv or pip
- Our current Python interface is run through a script (run_python_with_axom.sh) that adds the necessary dependencies to PYTHONPATH
- This PR adds another option to generate a scikit-build-core project over an installed Axom. The generated Python wheel can be installed into a uv or pip virtual environment
- It also adds a CI job to build and test the wheel via uv from an already built Axom installation (the gcc docker CI)
This PR also improves the Sidre bindings to (a) fix type helpers for better IDE integration and (b) fixes a potential memory leak related to pinning external views into memory already tracked by our interface. It adds several tests for this memory fix

Three steps to a uv installation

Assuming we already have an Axom installation in ${AXOM_INSTALL}

   # 1. create a uv virtual environment
   $ uv venv --python $(which python3)

   # 2. build the wheel against an Axom installation (internal logic adds the Conduit that axom was built against)
   $ uv pip install /path/to/axom/src/python \
       -C cmake.define.AXOM_DIR="$AXOM_INSTALL/lib/cmake"

   # 3. run an axom.sidre Python script through this environment
   $ uv run python -c "import axom.sidre, conduit, numpy; print(axom.__version__)"

Running in Jupyter w/ type hints

   # 1. install Jupyterlab, and optionally, extra language servers
   $ uv pip install jupyterlab ipykernel
   $ uv pip install jupyterlab-lsp 'python-lsp-server[all]'

   # 2. generate an Axom kernel
   $ uv run python -m ipykernel install --user --name axom --display-name "Axom (uv)"

   # 3. run jupyter
   $ uv run jupyter lab

Our Axom kernel has tab completion on the API:

It also has tooltips for the API (via shift+tab)

Somewhat larger example:

Limitations

Usage errors in Jupyter currently pipe through cerr (via SLIC_WARNING) and don't show up in the error Jupyter output cell. E.g if you run the cell to create a view or group a second time, it will silently get converted to None.
- We should use exceptions in the Python interface instead of slic errors/warnings. I'll address that in a separate PR. See Sidre's Python interface should throw errors instead of silently failing #1938
I didn't test this with hip or cuda configs. Let's also handle that in a separate PR

kennyweiss · 2026-08-01T00:33:45Z

This user doc about Python will move out of sidre once we have bindings for a second component

kennyweiss · 2026-08-01T00:34:51Z

 //! Erase all pins recorded for \a ds (called when the DataStore is collected).
 void releaseDataStoreExternalPins(DataStore* ds) { externalDataOwnerRegistry().erase(ds); }

+//! Release the pin recorded for \a view, if any (defined below).


This block contains fixes for pinning memory in blocks we've already pinned

kennyweiss · 2026-08-01T00:35:43Z


-  m_sidre.def("indexIsValid", &indexIsValid, "Returns true if idx is valid, else false.");
-  m_sidre.def("nameIsValid", &nameIsValid, "Returns true if name is valid, else false.");
+  m_sidre.def("indexIsValid",


The nb::arg() additions help Jupyter and IDEs get the names of functions (instead of just arg1, arg2

kennyweiss · 2026-08-01T00:36:27Z

-         "Create an Attribute object with a default string value")
+         "Create an Attribute object with a default string value",
+         nb::arg("name"),
+         nb::arg("default_value").noconvert())


The noconvert() prohibits casts like float -> int. (see block comment below in this file)

kennyweiss · 2026-08-01T00:38:21Z

This file helps us get the conduit that Axom was configured against

kennyweiss · 2026-08-01T00:40:36Z

This is the CMake project that uv uses to set up the wheel for uv.

I tried to make it as simple as possible, but there were some necessary complexities.
I'm not sure how thoroughly tests the stable ABI options are, but they seemed worthwhile.

kennyweiss · 2026-08-01T00:41:45Z

This pyproject.toml file sets up the wheel and its dependencies.

bmhan12 · 2026-08-18T17:20:31Z

+# The compiled extension and its generated _sidre.pyi are installed by CMakeLists.txt into axom/sidre/.
+wheel.packages = ["src/axom"]
+
+# NOTE on the sdist: the binding translation unit lives with its component (../axom/sidre/nanobind_sidre.cpp), outside this project directory.


is this comment block necessary, or is there setting(s) here that should be made optional?

bmhan12 · 2026-08-19T16:11:02Z

+   $ uv pip install /path/to/axom/src/python \
+       -C cmake.define.AXOM_DIR="$AXOM_INSTALL/lib/cmake"
+
+   $ uv run python -c "import axom.sidre, conduit, numpy; print(axom.__version__)"


Thanks for the documentation!

Here's the ReadTheDocs link for reference:
https://axom.readthedocs.io/en/feature-kweiss-python-env/axom/sidre/docs/sphinx/python_interface.html

I tried following along, just wanted to note that AXOM_DIR needs to be an absolute path.

Also, I tried the uv run python command here, and got a configuration error:

Click for dropdown of error

$ uv run python -c "import axom.sidre, conduit, numpy; print(axom.__version__)" × Failed to build `axom-pysidre @ file:///usr/WS1/han12/axom_codex` ├─▶ The build backend returned an error ╰─▶ Call to `scikit_build_core.build.build_editable` failed (exit status: 1) [stdout] *** scikit-build-core 1.0.3 using CMake 3.26.5 (editable) *** Configuring CMake... loading initial cache file build/cp313-cp313-linux_x86_64/CMakeInit.txt -- The C compiler identification is GNU 13.3.1 -- The CXX compiler identification is GNU 13.3.1 -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Check for working C compiler: /usr/tce/packages/gcc/gcc-13.3.1/bin/gcc - skipped -- Detecting C compile features -- Detecting C compile features - done -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Check for working CXX compiler: /usr/tce/packages/gcc/gcc-13.3.1/bin/g++ - skipped -- Detecting CXX compile features -- Detecting CXX compile features - done -- BLT Version: 0.7.2 -- CMake Version: 3.26.5 -- CMake Build Type: Release -- CMake Executable: /usr/bin/cmake -- Setting CMAKE_CXX_EXTENSIONS to OFF -- Standard C++20 selected -- BLT MPI support is OFF -- BLT OpenMP support is OFF -- BLT CUDA support is OFF -- BLT HIP support is OFF -- Found Git: /usr/tce/bin/git (found version "2.29.1") -- Git Support is ON -- Git Executable: /usr/tce/bin/git -- Git Version: 2.29.1 -- Sphinx support is OFF -- Valgrind support is OFF -- AStyle support is OFF -- ClangFormat support is OFF -- Uncrustify support is OFF -- Yapf support is OFF -- CMakeFormat support is OFF -- Cppcheck support is OFF -- ClangQuery support is OFF -- ClangTidy support is OFF -- ClangApplyReplacements support is OFF -- C Compiler family is GNU -- Adding optional BLT definitions and compiler flags -- Enabling all compiler warnings on all targets. -- Fortran support disabled. -- CMAKE_C_FLAGS flags are: -Wall -Wextra -- CMAKE_CXX_FLAGS flags are: -Wall -Wextra -- CMAKE_EXE_LINKER_FLAGS flags are: -- CMake's C implicit link directories: /collab/usr/global/tools/tce4/packages/gcc/gcc-13.3.1/lib/gcc/x86_64-redhat-linux/13;/collab/usr/global/tools/tce4/packages/gcc/gcc-13.3.1/lib/gcc;/collab/usr/global/tools/tce4/packages/gcc/gcc-13.3.1/lib64;/lib64;/usr/lib64;/collab/usr/global/tools/tce4/packages/gcc/gcc-13.3.1/lib -- CMake's CXX implicit link directories: /collab/usr/global/tools/tce4/packages/gcc/gcc-13.3.1/lib/gcc/x86_64-redhat-linux/13;/collab/usr/global/tools/tce4/packages/gcc/gcc-13.3.1/lib/gcc;/collab/usr/global/tools/tce4/packages/gcc/gcc-13.3.1/lib64;/lib64;/usr/lib64;/collab/usr/global/tools/tce4/packages/gcc/gcc-13.3.1/lib -- CMake's C implicit link libraries: gcc;gcc_s;c;gcc;gcc_s -- CMake's CXX implicit link libraries: stdc++;m;gcc_s;gcc;c;gcc_s;gcc -- Umpire support is OFF -- RAJA support is OFF -- Conduit support is OFF -- HDF5 support is OFF -- MFEM support is OFF -- Adiak support is OFF -- Caliper support is OFF -- Shroud support is OFF -- Found Python: /g/g12/han12/.cache/uv/builds-v0/.tmptcFZjg/bin/python (found suitable version "3.13.11", minimum required is "3.8") found components: Interpreter Development.Module -- Python version: 3.13.11 -- Python executable: /g/g12/han12/.cache/uv/builds-v0/.tmptcFZjg/bin/python -- Python include dir: /collab/usr/gapps/axom/devtools/toss_4_x86_64_ib/2026_02_17_14_42_14/gcc-13.3.1/python-3.13.11-unvqpdaz447knce6gej3tdldeqqsdssl/include/python3.13 -- Python library: -- Configuring incomplete, errors occurred! [stderr] WARNING: Minimum CMake version not found in CMakeLists.txt. If the CMakeLists.txt is valid, this might be a bug in our search algorithm. Report this or (and) set manually to avoid this warning. CMake Error at cmake/thirdparty/SetupAxomThirdParty.cmake:409 (message): Axom's python extensions require conduit and numpy at runtime. The python library installation paths can be specified with CMake variables: CONDUIT_PYTHON_MODULE_DIR, PY_NUMPY_DIR Call Stack (most recent call first): cmake/CMakeBasics.cmake:21 (include) CMakeLists.txt:128 (include) *** CMake configuration failed

I was able to resolve it by adding --no-sync flag, not sure if that's expected.

This directory is used by the Python installation (e.g. for wheels).

Also deemphasizes the discussion of platform-dependent wheelhouses.

This allows for better IDE integration.

And about better IDE integration with Jupyter.

Only export variables with content, and clarify the intent of the two variable groups.

Do not pin pointers within blocks that are already pinned.

…f noconvert()

Allow for the possibility of some generated config files not being present and return a proper error message.

Misc: Slight fixes to docs.

…om install

…NDARD as Axom

kennyweiss requested review from Arlie-Capps, BradWhitlock, bmhan12, cyrush, jcs15c, nselliott, publixsubfan, rhornung67 and white238 August 1, 2026 00:30

kennyweiss self-assigned this Aug 1, 2026

kennyweiss added enhancement New feature or request Sidre Issues related to Axom's 'sidre' component CI Issues related to continuous integration Documentation Issues related to documentation usability Related to code usability Python labels Aug 1, 2026

kennyweiss commented Aug 1, 2026

View reviewed changes

Comment thread src/python/pyproject.toml

kennyweiss Aug 1, 2026

Copy link
Copy Markdown

Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This pyproject.toml file sets up the wheel and its dependencies.

kennyweiss commented Aug 1, 2026

View reviewed changes

Comment thread .github/workflows/ci-tests.yml Outdated

kennyweiss force-pushed the feature/kweiss/python-env branch from 0f6539b to 9323289 Compare August 2, 2026 19:50

This was referenced Aug 2, 2026

Add uv to our docker CI images #1937

Open

Sidre's Python interface should throw errors instead of silently failing #1938

Open

kennyweiss force-pushed the feature/kweiss/python-env branch from 9323289 to 82fb536 Compare August 7, 2026 23:42

kennyweiss force-pushed the feature/kweiss/python-env branch from 5ee9523 to f0f3639 Compare August 19, 2026 04:39

bmhan12 reviewed Aug 19, 2026

View reviewed changes

kennyweiss added 29 commits August 27, 2026 18:34

Sidre/Python: Improves lifetime tests

68b79af

Adds build/ to .gitignore

a653809

This directory is used by the Python installation (e.g. for wheels).

Python: Streamlines Python installation docs

9c60942

Python: Tests generate files, so run run from a scratch directory

6993439

Minor touchups to docs

d5519d4

Python: Fixes and streamlines setup for uv/pip

ac66160

Python: Use AXOM_DIR instead of axom_DIR

bf1dad5

Also deemphasizes the discussion of platform-dependent wheelhouses.

Python: More streamlining of docs

b90dec1

Sidre/Python: Annotate and name sidre binding function arguments

b11a569

This allows for better IDE integration.

Python: Improves docs about installing optional dependencies

f4cb4b4

And about better IDE integration with Jupyter.

Python: Updates Python docker CI script to match docs

aaef4d1

Python: More consolidation of Python setup/installation docs

d1201a1

Python: Clarifies error message and docs associate with stable API

3cad60b

Python: Improves logic for finding the conduit python module

8e46b46

Python: Resolve conduit python location to an absolute path

fcaa276

Python: Fixes wheel lookup of Conduit Python path

e78c007

Python: Remove untested HIP workaround for uv bindings

2f38547

Python: Clarify and fix env shell script

28fd116

Only export variables with content, and clarify the intent of the two variable groups.

Sidre/Python: Improves pinning logic for external buffers

f2f5efb

Do not pin pointers within blocks that are already pinned.

Sidre/Python: Adds tests and clarifying comment about repurcussions o…

5379dfa

…f noconvert()

Python: Handle differences between two python installation paths

a6c69a8

Allow for the possibility of some generated config files not being present and return a proper error message.

Python: Use set -e and set -x instead of or_die in wheels CI script

4516535

Misc: Slight fixes to docs.

Python: Ensure our Python gets MPI when Axom is configured with MPI

8ad35f0

Python: Fixes CI to work locally

8082cfb

Python: Some minor fixups and clarifications

93e4c31

CI bugfix -- need to run the wheels test through a job that has an Ax…

747ee1e

…om install

CI bugfix -- ensure we can install uv in our CI images

573061b

Python: Ensures the python bindings project is using the same CXX_STA…

427fcad

…NDARD as Axom

Updates RELEASE-NOTES

8f67e36

kennyweiss force-pushed the feature/kweiss/python-env branch from f0f3639 to 8f67e36 Compare August 28, 2026 01:41

Navigation Menu

Uh oh!

Enable Axom's Python to be installed via uv/pip - #1936

Enable Axom's Python to be installed via uv/pip#1936
kennyweiss wants to merge 35 commits into
developfrom
feature/kweiss/python-env

kennyweiss commented Aug 1, 2026 •

edited

Loading

Uh oh!

kennyweiss Aug 1, 2026

Uh oh!

kennyweiss Aug 1, 2026

Uh oh!

kennyweiss Aug 1, 2026

Uh oh!

kennyweiss Aug 1, 2026 •

edited

Loading

Uh oh!

kennyweiss Aug 1, 2026

Uh oh!

kennyweiss Aug 1, 2026

Uh oh!

kennyweiss Aug 1, 2026

Uh oh!

Uh oh!

bmhan12 Aug 18, 2026

Uh oh!

bmhan12 Aug 19, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

kennyweiss commented Aug 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Three steps to a uv installation

Running in Jupyter w/ type hints

Limitations

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kennyweiss Aug 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

kennyweiss commented Aug 1, 2026 •

edited

Loading

kennyweiss Aug 1, 2026 •

edited

Loading