Skip to content

Add CUDA 12.9 Build Variants#489

Open
mharradon wants to merge 2 commits into
conda-forge:mainfrom
mharradon:CUDA129
Open

Add CUDA 12.9 Build Variants#489
mharradon wants to merge 2 commits into
conda-forge:mainfrom
mharradon:CUDA129

Conversation

@mharradon
Copy link
Copy Markdown

@mharradon mharradon commented May 13, 2026 •
edited
Loading

Re-rendered with conda-smithy 3.61.2 and conda-forge-pinning 2026.05.12.12.28.02

Checklist

  • Used a personal fork of the feedstock to propose changes
  • Bumped the build number (if the version is unchanged)
  • Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
  • Ensured the license file is being packaged.

Closes #487

@mharradon mharradon mentioned this pull request May 13, 2026
Open
@conda-forge-admin
Copy link
Copy Markdown
Contributor

conda-forge-admin commented May 13, 2026

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe/recipe.yaml) and found it was in an excellent condition.

@mharradon mharradon changed the title Add CUDA 12.9 build variants, Add CUDA 12.9 build Variants May 13, 2026
@mharradon mharradon changed the title Add CUDA 12.9 build Variants Add CUDA 12.9 Build Variants May 13, 2026
@mharradon
Copy link
Copy Markdown
Author

mharradon commented May 16, 2026 •
edited
Loading

Note aarch64 and osx_64 are already broken on main - the working builds all pass.

@hmaarrfk
Copy link
Copy Markdown
Contributor

hmaarrfk commented May 16, 2026

Can you confirm that you've tested this locally? i've had to revert the cuda 12.9 build in the past.

@mharradon
Copy link
Copy Markdown
Author

mharradon commented May 16, 2026

Ah, good question then...

Last night I built linux_64_cuda_compiler_version12.9microarch_level1 to test:

python build-locally.py linux_64_cuda_compiler_version12.9microarch_level1.

I then created an environment just with that - solving to:

List of packages in environment: "/home/mharradon/miniconda3/envs/tf-cuda129-test"

  Name                     Version       Build                     Channel
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
  _openmp_mutex            4.5           20_gnu                    conda-forge
  _x86_64-microarch-level  3             3_x86_64_v3               conda-forge
  absl-py                  2.4.0         pyhd8ed1ab_0              conda-forge
  astunparse               1.6.3         pyhd8ed1ab_3              conda-forge
  aws-c-auth               0.10.1        ha62d5e7_3                conda-forge
  aws-c-cal                0.9.13        h2c9d079_1                conda-forge
  aws-c-common             0.12.6        hb03c661_0                conda-forge
  aws-c-compression        0.3.2         h8b1a151_0                conda-forge
  aws-c-http               0.10.13       h4bacb7b_0                conda-forge
  aws-c-io                 0.26.3        h692f434_1                conda-forge
  aws-c-s3                 0.12.2        he6ee468_1                conda-forge
  aws-c-sdkutils           0.2.4         h8b1a151_4                conda-forge
  aws-checksums            0.2.10        h8b1a151_0                conda-forge
  backports.zstd           1.5.0         py311h6b1f9c4_0           conda-forge
  brotli-python            1.2.0         py311h66f275b_1           conda-forge
  bzip2                    1.0.8         hda65f42_9                conda-forge
  c-ares                   1.34.6        hb03c661_0                conda-forge
  ca-certificates          2026.4.22     hbd8a1cb_0                conda-forge
  cached-property          1.5.2         hd8ed1ab_1                conda-forge
  cached_property          1.5.2         pyha770c72_1              conda-forge
  certifi                  2026.4.22     pyhd8ed1ab_0              conda-forge
  charset-normalizer       3.4.7         pyhd8ed1ab_0              conda-forge
  cuda-crt-tools           12.9.86       ha770c72_2                conda-forge
  cuda-cudart              12.9.79       h5888daf_0                conda-forge
  cuda-cudart_linux-64     12.9.79       h3f2d84a_0                conda-forge
  cuda-cupti               12.9.79       h676940d_1                conda-forge
  cuda-nvcc-tools          12.9.86       he02047a_2                conda-forge
  cuda-nvrtc               12.9.86       hecca717_1                conda-forge
  cuda-nvtx                12.9.79       hecca717_1                conda-forge
  cuda-nvvm-tools          12.9.86       h4bc722e_2                conda-forge
  cuda-version             12.9          h4f385c5_3                conda-forge
  cudnn                    9.10.2.21     hbcb9cd8_0                conda-forge
  flatbuffers              25.9.23       hb7d4c21_0                conda-forge
  gast                     0.7.0         pyhd8ed1ab_0              conda-forge
  giflib                   5.2.2         hd590300_0                conda-forge
  google-pasta             0.2.0         pyhd8ed1ab_2              conda-forge
  grpcio                   1.78.1        py311h3aa0767_0           conda-forge
  h2                       4.3.0         pyhcf101f3_0              conda-forge
  h5py                     3.16.0        nompi_py311hfef529e_102   conda-forge
  hdf5                     2.1.0         nompi_h87a9417_105        conda-forge
  hpack                    4.1.0         pyhd8ed1ab_0              conda-forge
  hyperframe               6.1.0         pyhd8ed1ab_0              conda-forge
  icu                      78.3          h33c6efd_0                conda-forge
  idna                     3.13          pyhcf101f3_0              conda-forge
  importlib-metadata       8.8.0         pyhcf101f3_0              conda-forge
  keras                    3.14.1        pyh753f3f9_0              conda-forge
  keyutils                 1.6.3         hb9d3cd8_0                conda-forge
  krb5                     1.22.2        ha1258a1_0                conda-forge
  ld_impl_linux-64         2.45.1        default_hbd61a6d_102      conda-forge
  libabseil                20260107.1    cxx17_h7b12aa8_0          conda-forge
  libaec                   1.1.5         h088129d_0                conda-forge
  libblas                  3.11.0        7_h4a7cf45_openblas       conda-forge
  libcblas                 3.11.0        7_h0358290_openblas       conda-forge
  libcublas                12.9.2.10     h676940d_0                conda-forge
  libcudnn                 9.10.2.21     hf7e9902_0                conda-forge
  libcudnn-dev             9.10.2.21     h58dd1b1_0                conda-forge
  libcufft                 11.4.1.4      hecca717_1                conda-forge
  libcurand                10.3.10.19    h676940d_1                conda-forge
  libcurl                  8.20.0        hcf29cc6_0                conda-forge
  libcusolver              11.7.5.82     h676940d_2                conda-forge
  libcusparse              12.5.10.65    hecca717_2                conda-forge
  libedit                  3.1.20250104  pl5321h7949ede_0          conda-forge
  libev                    4.33          hd590300_2                conda-forge
  libexpat                 2.8.0         hecca717_0                conda-forge
  libffi                   3.5.2         h3435931_0                conda-forge
  libgcc                   15.2.0        he0feb66_19               conda-forge
  libgcc-ng                15.2.0        h69a702a_19               conda-forge
  libgfortran              15.2.0        h69a702a_19               conda-forge
  libgfortran5             15.2.0        h68bc16d_19               conda-forge
  libgomp                  15.2.0        he0feb66_19               conda-forge
  libgrpc                  1.78.1        h1d1128b_0                conda-forge
  libjpeg-turbo            3.1.4.1       hb03c661_0                conda-forge
  liblapack                3.11.0        7_h47877c9_openblas       conda-forge
  liblzma                  5.8.3         hb03c661_0                conda-forge
  libml_dtypes-headers     0.5.4         h707e725_0                conda-forge
  libnghttp2               1.68.1        h877daf1_0                conda-forge
  libnsl                   2.0.1         hb9d3cd8_1                conda-forge
  libnvjitlink             12.9.86       hecca717_2                conda-forge
  libopenblas              0.3.33        pthreads_h94d23a6_0       conda-forge
  libpng                   1.6.58        h421ea60_0                conda-forge
  libprotobuf              6.33.5        h2b00c02_0                conda-forge
  libre2-11                2025.11.05    h0dc7533_1                conda-forge
  libsqlite                3.53.1        h0c1763c_0                conda-forge
  libssh2                  1.11.1        hcf80075_0                conda-forge
  libstdcxx                15.2.0        h934c35e_19               conda-forge
  libtensorflow_cc         2.19.1        cuda129h92c9f35_206       file:///home/mharradon/tensorflow-feedstock/build_artifacts
  libtensorflow_framework  2.19.1        cuda129hf6c7d65_206       file:///home/mharradon/tensorflow-feedstock/build_artifacts
  libuuid                  2.42          h5347b49_0                conda-forge
  libxcrypt                4.4.36        hd590300_1                conda-forge
  libzlib                  1.3.2         h25fd6f3_2                conda-forge
  markdown                 3.10.2        pyhcf101f3_0              conda-forge
  markdown-it-py           4.2.0         pyhd8ed1ab_0              conda-forge
  markupsafe               3.0.3         py311h3778330_1           conda-forge
  mdurl                    0.1.2         pyhd8ed1ab_1              conda-forge
  ml_dtypes                0.5.4         np2py311h912ec1f_1        conda-forge
  namex                    0.1.0         pyhd8ed1ab_0              conda-forge
  nccl                     2.30.4.1      h4d09622_0                conda-forge
  ncurses                  6.6           hdb14827_0                conda-forge
  numpy                    2.4.5         py311h2e04523_0           conda-forge
  openssl                  3.6.2         h35e630c_0                conda-forge
  opt_einsum               3.4.0         pyhd8ed1ab_1              conda-forge
  optree                   0.19.1        py311hdf67eae_0           conda-forge
  packaging                26.2          pyhc364b38_0              conda-forge
  pip                      26.1.1        pyh8b19718_0              conda-forge
  protobuf                 6.33.5        py311h3f0a9aa_2           conda-forge
  pygments                 2.20.0        pyhd8ed1ab_0              conda-forge
  pysocks                  1.7.1         pyha55dd90_7              conda-forge
  python                   3.11.15       hd63d673_0_cpython        conda-forge
  python-flatbuffers       25.9.23       pyh1e1bc0e_0              conda-forge
  python_abi               3.11          8_cp311                   conda-forge
  re2                      2025.11.05    h5301d42_1                conda-forge
  readline                 8.3           h853b02a_0                conda-forge
  requests                 2.34.2        pyhcf101f3_0              conda-forge
  rich                     15.0.0        pyhcf101f3_0              conda-forge
  s2n                      1.7.2         hc5a330e_1                conda-forge
  setuptools               82.0.1        pyh332efcf_0              conda-forge
  six                      1.17.0        pyhe01879c_1              conda-forge
  snappy                   1.2.2         h03e3b7b_1                conda-forge
  tensorboard              2.19.0        pyhd8ed1ab_0              conda-forge
  tensorboard-data-server  0.7.0         py311h97c413e_4           conda-forge
  tensorflow               2.19.1        cuda129py311h9af8fa9_206  file:///home/mharradon/tensorflow-feedstock/build_artifacts
  tensorflow-base          2.19.1        cuda129py311hf58c0f9_206  file:///home/mharradon/tensorflow-feedstock/build_artifacts
  tensorflow-gpu           2.19.1        cuda129h53ccfdb_206       file:///home/mharradon/tensorflow-feedstock/build_artifacts
  termcolor                3.3.0         pyhd8ed1ab_0              conda-forge
  tk                       8.6.13        noxft_h366c992_103        conda-forge
  typing-extensions        4.15.0        h396c80c_0                conda-forge
  typing_extensions        4.15.0        pyhcf101f3_0              conda-forge
  tzdata                   2025c         hc9c84f9_1                conda-forge
  urllib3                  2.7.0         pyhd8ed1ab_0              conda-forge
  werkzeug                 3.1.8         pyhcf101f3_0              conda-forge
  wheel                    0.47.0        pyhd8ed1ab_0              conda-forge
  wrapt                    2.1.2         py311h49ec1c0_0           conda-forge
  zipp                     3.23.1        pyhcf101f3_0              conda-forge
  zstd                     1.5.7         hb78ec9c_6                conda-forge

After that I ran a MNIST training test. In nvidia-smi I observed expected GPU utilization, and the test trained successfully. I also spot-checked a few basic tensor arithmetic operations and the values were correct.

This was all carried out in WSL on my GeForce RTX 4070 Ti, Driver Version: 591.59.

@hmaarrfk please let me know if you think any additional testing or info would be helpful!

@hmaarrfk
Copy link
Copy Markdown
Contributor

hmaarrfk commented May 16, 2026

I"m a little scared of the backport to be honest. I would somewhat rather updating the version of tensorflow (though that is very hard i agree).

Without a full elaborate test, i'm affraid to see similar failures that me revert the addition before.

@hmaarrfk
Copy link
Copy Markdown
Contributor

hmaarrfk commented May 16, 2026

and i somewhat don't trust WSL.

If an other maintainer wants to merge, that might be helpful

@mharradon
Copy link
Copy Markdown
Author

mharradon commented May 16, 2026

That's fine - though I'm not sure I'm tracking what's a "backport" etc.

All that Google reports testing for TF 2.18.0 through 2.21.0 is CUDA 12.5, so anything other than that seems risky. I'm also not aware of official plans by Google to support CUDA 13.0.

If there's specific testing you can point me to I can try that. I'm trying to run the TensorFlow test suite now.

My only goal is to get recent TensorFlow + Jax + CUDA in a single environment, so it would also work to add CUDA 12.8 builds to Jax - I figured adding older CUDA versions would be less welcome.

If the intent is to get the whole conda-forge ecosystem on CUDA 13 in the next few months that could be fine too. From the communications I can see it didn't look like that was any more promising for TensorFlow than 12.9 (I would assume the major version bump would be even more challenging).

@mharradon mharradon mentioned this pull request May 17, 2026
Open
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@FarhanTejani FarhanTejani Awaiting requested review from FarhanTejani FarhanTejani is a code owner

@ghego ghego Awaiting requested review from ghego ghego is a code owner

@h-vetinari h-vetinari Awaiting requested review from h-vetinari h-vetinari is a code owner

@hajapy hajapy Awaiting requested review from hajapy hajapy is a code owner

@hmaarrfk hmaarrfk Awaiting requested review from hmaarrfk hmaarrfk is a code owner

@isuruf isuruf Awaiting requested review from isuruf isuruf is a code owner

@jschueller jschueller Awaiting requested review from jschueller jschueller is a code owner

@ngam ngam Awaiting requested review from ngam ngam is a code owner

@njzjz njzjz Awaiting requested review from njzjz njzjz is a code owner

@waitingkuo waitingkuo Awaiting requested review from waitingkuo waitingkuo is a code owner

@wolfv wolfv Awaiting requested review from wolfv wolfv is a code owner

@xhochy xhochy Awaiting requested review from xhochy xhochy is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Request: CUDA 12.9 (Linux) Builds?

3 participants

@mharradon @conda-forge-admin @hmaarrfk