Solution to issue cannot be found in the documentation.
Issue
I created a fresh new environment:
conda create -n testcompile -c conda-forge python=3.12.* pytorch-gpu torchvision cuda-compiler
I deleted cache folders - /tmp/torchinductor_jj and /home/jj/distrobox/fedora/.triton/
So I have pytorch and CUDA works in it. But when I run any torch.compile-d function, on cpu or cuda, it causes the following exception:
BackendCompilerFailed: backend='inductor' raised:
ImportError: /var/home/jj/distrobox/fedora/.triton/cache/q4oIpkjOtdHHfi8xBkm4jC4JWIk5AjKtN8WRkZb8MD8/cuda_utils.so: undefined symbol: cuModuleGetFunction
Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information
for example I run this and it causes the exception:
import site
import torch
print(f'{site.getsitepackages() = }') # make sure corrent env is selected
print(f'{torch.cuda.is_available() = }') # check if CUDA is enabled
@torch.compile
def square(x):
return x ** 2
x = torch.randn(1, device='cpu')
square(x)
I also tried
conda create -n testcompile2 -c conda-forge python=3.12.* pytorch-gpu torchvision cuda-compiler=12.6
but I get the same "cuModuleGetFunction" error.
torch.compile does work on pip version of torch (but I have issues with some other unrelated packages on it). I also tried doing this in ubuntu distrobox instead of fedora, and got the same issue.
Installed packages
# packages in environment at /var/home/jj/distrobox/fedora/miniconda3/envs/testcompile:
#
# Name Version Build Channel
_openmp_mutex 4.5 3_kmp_llvm conda-forge
attr 2.5.1 h166bdaf_1 conda-forge
binutils 2.43 h4852527_4 conda-forge
binutils_impl_linux-64 2.43 h4bf12b8_4 conda-forge
binutils_linux-64 2.43 h4852527_4 conda-forge
bzip2 1.0.8 h4bc722e_7 conda-forge
c-compiler 1.9.0 h2b85faf_0 conda-forge
ca-certificates 2025.1.31 hbcca054_0 conda-forge
cpython 3.12.9 py312hd8ed1ab_1 conda-forge
cuda-cccl_linux-64 12.8.90 ha770c72_1 conda-forge
cuda-compiler 12.8.1 hbad6d8a_0 conda-forge
cuda-crt-dev_linux-64 12.8.93 ha770c72_0 conda-forge
cuda-crt-tools 12.8.93 ha770c72_0 conda-forge
cuda-cudart 12.8.90 h5888daf_1 conda-forge
cuda-cudart-dev 12.8.90 h5888daf_1 conda-forge
cuda-cudart-dev_linux-64 12.8.90 h3f2d84a_1 conda-forge
cuda-cudart-static 12.8.90 h5888daf_1 conda-forge
cuda-cudart-static_linux-64 12.8.90 h3f2d84a_1 conda-forge
cuda-cudart_linux-64 12.8.90 h3f2d84a_1 conda-forge
cuda-cuobjdump 12.8.90 hbd13f7d_0 conda-forge
cuda-cupti 12.8.90 hbd13f7d_0 conda-forge
cuda-cuxxfilt 12.8.90 hbd13f7d_0 conda-forge
cuda-driver-dev_linux-64 12.8.90 h3f2d84a_1 conda-forge
cuda-nvcc 12.8.93 hcdd1206_0 conda-forge
cuda-nvcc-dev_linux-64 12.8.93 he91c749_0 conda-forge
cuda-nvcc-impl 12.8.93 h85509e4_0 conda-forge
cuda-nvcc-tools 12.8.93 he02047a_0 conda-forge
cuda-nvcc_linux-64 12.8.93 h04802cd_0 conda-forge
cuda-nvdisasm 12.8.90 hbd13f7d_0 conda-forge
cuda-nvprune 12.8.90 hbd13f7d_0 conda-forge
cuda-nvrtc 12.8.93 hbd13f7d_0 conda-forge
cuda-nvtx 12.8.90 hbd13f7d_0 conda-forge
cuda-nvvm-dev_linux-64 12.8.93 ha770c72_0 conda-forge
cuda-nvvm-impl 12.8.93 he02047a_0 conda-forge
cuda-nvvm-tools 12.8.93 he02047a_0 conda-forge
cuda-version 12.8 h5d125a7_3 conda-forge
cudnn 9.8.0.87 h81d5506_0 conda-forge
cusparselt 0.7.0.0 hcd2ec93_0 conda-forge
cxx-compiler 1.9.0 h1a2810e_0 conda-forge
filelock 3.17.0 pyhd8ed1ab_0 conda-forge
freetype 2.12.1 h267a509_2 conda-forge
fsspec 2025.3.0 pyhd8ed1ab_0 conda-forge
gcc 13.3.0 h9576a4e_2 conda-forge
gcc_impl_linux-64 13.3.0 h1e990d8_2 conda-forge
gcc_linux-64 13.3.0 hc28eda2_8 conda-forge
giflib 5.2.2 hd590300_0 conda-forge
gmp 6.3.0 hac33072_2 conda-forge
gmpy2 2.1.5 py312h7201bc8_3 conda-forge
gxx 13.3.0 h9576a4e_2 conda-forge
gxx_impl_linux-64 13.3.0 hae580e1_2 conda-forge
gxx_linux-64 13.3.0 h6834431_8 conda-forge
icu 75.1 he02047a_0 conda-forge
jinja2 3.1.6 pyhd8ed1ab_0 conda-forge
kernel-headers_linux-64 3.10.0 he073ed8_18 conda-forge
lcms2 2.17 h717163a_0 conda-forge
ld_impl_linux-64 2.43 h712a8e2_4 conda-forge
lerc 4.0.0 h27087fc_0 conda-forge
libabseil 20240722.0 cxx17_hbbce691_4 conda-forge
libblas 3.9.0 31_hfdb39a5_mkl conda-forge
libcap 2.71 h39aace5_0 conda-forge
libcblas 3.9.0 31_h372d94f_mkl conda-forge
libcublas 12.8.4.1 h9ab20c4_0 conda-forge
libcudss0 0.4.0.2 he55f5cd_2 conda-forge
libcufft 11.3.3.83 hbd13f7d_0 conda-forge
libcufile 1.13.1.3 h12f29b5_0 conda-forge
libcurand 10.3.9.90 hbd13f7d_0 conda-forge
libcusolver 11.7.3.90 h9ab20c4_0 conda-forge
libcusparse 12.5.8.93 hbd13f7d_0 conda-forge
libdeflate 1.23 h4ddbbb0_0 conda-forge
libexpat 2.6.4 h5888daf_0 conda-forge
libffi 3.4.6 h2dba641_0 conda-forge
libgcc 14.2.0 h767d61c_2 conda-forge
libgcc-devel_linux-64 13.3.0 hc03c837_102 conda-forge
libgcc-ng 14.2.0 h69a702a_2 conda-forge
libgcrypt-lib 1.11.0 hb9d3cd8_2 conda-forge
libgomp 14.2.0 h767d61c_2 conda-forge
libgpg-error 1.51 hbd13f7d_1 conda-forge
libhwloc 2.11.2 default_h0d58e46_1001 conda-forge
libiconv 1.18 h4ce23a2_1 conda-forge
libjpeg-turbo 3.0.0 hd590300_1 conda-forge
liblapack 3.9.0 31_hc41d3b0_mkl conda-forge
libllvm20 20.1.0.rc1 hb8a341e_0 conda-forge
liblzma 5.6.4 hb9d3cd8_0 conda-forge
libmagma 2.8.0 h566cb83_2 conda-forge
libnl 3.11.0 hb9d3cd8_0 conda-forge
libnsl 2.0.1 hd590300_0 conda-forge
libnvjitlink 12.8.93 hbd13f7d_0 conda-forge
libnvjpeg 12.3.5.92 h97fd463_0 conda-forge
libpng 1.6.47 h943b412_0 conda-forge
libprotobuf 5.28.3 h6128344_1 conda-forge
libsanitizer 13.3.0 he8ea267_2 conda-forge
libsqlite 3.49.1 hee588c1_1 conda-forge
libstdcxx 14.2.0 h8f9b012_2 conda-forge
libstdcxx-devel_linux-64 13.3.0 hc03c837_102 conda-forge
libstdcxx-ng 14.2.0 h4852527_2 conda-forge
libsystemd0 257.3 h3dc2cb9_0 conda-forge
libtiff 4.7.0 hd9ff511_3 conda-forge
libtorch 2.6.0 cuda126_mkl_h8247c52_300 conda-forge
libudev1 257.3 h9a4d06a_0 conda-forge
libuuid 2.38.1 h0b41bf4_0 conda-forge
libuv 1.50.0 hb9d3cd8_0 conda-forge
libwebp-base 1.5.0 h851e524_0 conda-forge
libxcb 1.17.0 h8a09558_0 conda-forge
libxcrypt 4.4.36 hd590300_1 conda-forge
libxml2 2.13.6 h8d12d68_0 conda-forge
libzlib 1.3.1 hb9d3cd8_2 conda-forge
llvm-openmp 19.1.7 h024ca30_0 conda-forge
lz4-c 1.10.0 h5888daf_1 conda-forge
markupsafe 3.0.2 py312h178313f_1 conda-forge
mkl 2024.2.2 ha957f24_16 conda-forge
mpc 1.3.1 h24ddda3_1 conda-forge
mpfr 4.2.1 h90cbb55_3 conda-forge
mpmath 1.3.0 pyhd8ed1ab_1 conda-forge
nccl 2.25.1.1 ha44e49d_0 conda-forge
ncurses 6.5 h2d0b736_3 conda-forge
networkx 3.4.2 pyh267e887_2 conda-forge
numpy 2.2.3 py312h72c5963_0 conda-forge
openjpeg 2.5.3 h5fbd93e_0 conda-forge
openssl 3.4.1 h7b32b05_0 conda-forge
optree 0.14.1 py312h68727a3_0 conda-forge
pillow 11.1.0 py312h80c1187_0 conda-forge
pip 25.0.1 pyh8b19718_0 conda-forge
pthread-stubs 0.4 hb9d3cd8_1002 conda-forge
pybind11 2.13.6 pyh1ec8472_2 conda-forge
pybind11-global 2.13.6 pyh415d2e4_2 conda-forge
python 3.12.9 h9e4cc4f_1_cpython conda-forge
python_abi 3.12 5_cp312 conda-forge
pytorch 2.6.0 cuda126_mkl_py312_h8bb5dc9_300 conda-forge
pytorch-gpu 2.6.0 cuda126_mkl_ha999a5f_300 conda-forge
rdma-core 56.0 h5888daf_0 conda-forge
readline 8.2 h8c095d6_2 conda-forge
setuptools 75.8.2 pyhff2d567_0 conda-forge
sleef 3.8 h1b44611_0 conda-forge
sympy 1.13.3 pyh2585a3b_105 conda-forge
sysroot_linux-64 2.17 h0157908_18 conda-forge
tbb 2021.13.0 hceb3a55_1 conda-forge
tk 8.6.13 noxft_h4845f30_101 conda-forge
torchvision 0.21.0 cuda126_py312_h3e02b30_0 conda-forge
triton 3.2.0 cuda126py312h5a3d8a8_0 conda-forge
typing-extensions 4.12.2 hd8ed1ab_1 conda-forge
typing_extensions 4.12.2 pyha770c72_1 conda-forge
tzdata 2025a h78e105d_0 conda-forge
wheel 0.45.1 pyhd8ed1ab_1 conda-forge
xorg-libxau 1.0.12 hb9d3cd8_0 conda-forge
xorg-libxdmcp 1.1.5 hb9d3cd8_0 conda-forge
zstd 1.5.7 hb8e6e7a_1 conda-forge
Environment info
active environment : testcompile
active env location : /var/home/jj/distrobox/fedora/miniconda3/envs/testcompile
shell level : 2
user config file : /var/home/jj/distrobox/fedora/.condarc
populated config files : /var/home/jj/distrobox/fedora/miniconda3/.condarc
/var/home/jj/distrobox/fedora/.condarc
conda version : 25.1.1
conda-build version : not installed
python version : 3.12.8.final.0
solver : libmamba (default)
virtual packages : __archspec=1=skylake
__conda=25.1.1=0
__cuda=12.8=0
__glibc=2.40=0
__linux=6.13.5=0
__unix=0=0
base environment : /var/home/jj/distrobox/fedora/miniconda3 (writable)
conda av data dir : /var/home/jj/distrobox/fedora/miniconda3/etc/conda
conda av metadata url : None
channel URLs : https://conda.anaconda.org/nvidia/linux-64
https://conda.anaconda.org/nvidia/noarch
https://repo.anaconda.com/pkgs/main/linux-64
https://repo.anaconda.com/pkgs/main/noarch
https://repo.anaconda.com/pkgs/r/linux-64
https://repo.anaconda.com/pkgs/r/noarch
https://conda.anaconda.org/conda-forge/linux-64
https://conda.anaconda.org/conda-forge/noarch
package cache : /var/home/jj/distrobox/fedora/miniconda3/pkgs
/var/home/jj/distrobox/fedora/.conda/pkgs
envs directories : /var/home/jj/distrobox/fedora/miniconda3/envs
/var/home/jj/distrobox/fedora/.conda/envs
platform : linux-64
user-agent : conda/25.1.1 requests/2.32.3 CPython/3.12.8 Linux/6.13.5-102.bazzite.fc41.x86_64 fedora/41 glibc/2.40 solver/libmamba conda-libmamba-solver/25.1.1 libmambapy/2.0.5 aau/0.6.1 c/. s/. e/.
UID:GID : 1000:1000
netrc file : None
offline mode : False
Solution to issue cannot be found in the documentation.
Issue
I created a fresh new environment:
I deleted cache folders -
/tmp/torchinductor_jjand/home/jj/distrobox/fedora/.triton/So I have pytorch and CUDA works in it. But when I run any torch.compile-d function, on cpu or cuda, it causes the following exception:
for example I run this and it causes the exception:
I also tried
but I get the same "cuModuleGetFunction" error.
torch.compile does work on pip version of torch (but I have issues with some other unrelated packages on it). I also tried doing this in ubuntu distrobox instead of fedora, and got the same issue.
Installed packages
Environment info