Can not use Memtorch CUDA on Windows 10

[McLemoreT]

This issue seems very similar to #103 but I was unable to fix my problem doing the same thing as him. I am having a very similar problem. I went through the steps that the poster from #103 went through to get it working but I am still having issues. I have attached a log, this is what I get when I try to run "pip install memtorch". It mentions that it could not find the CUDA runtime, but as you can see in my other 2 screenshots, I clearly have it on my path and in the designated folder.
Memtorch_pip_log.txt

[coreylammie]

Hi @McLemoreT and @RagingFlames,

Apologies for the late response! I have self-assigned this issue, so we can discuss it here, rather than re-opening #103. Could you both please confirm that torch is able to identify your GPU, and specify your torch CUDA version using:

import torch

if torch.cuda.is_available():
    print(torch.version.cuda)

In addition, could you please check whether cl.exe is on PATH, i.e., you can run the command cl in a command prompt.

My suspicion is that this error is caused by a mismatch between your torch and CUDA Toolkit versions. If this is the case, I will add a disclaimer to install instructions that these must be the same. If these are are the same, and cl.exe is on PATH, could you please indicate your specific Python configuration so I can look into this further?

Kind Regards,

Corey.

[McLemoreT]

Thank you for getting back to me. I have no expectations for you to be waiting on the issues page for this project 24/7, especially since you seem to be the sole developer for the project. A one week turn around on the issue is pretty good in my book.

As for the issue, I have tried the things you have listed here. Running your python script yields no output, so it seems torch is unable to identify my GPU. I have one potential guess for that, being that I have two GPU's. I have an AMD 6700xt that I use generally, and I have a gtx 970 that I use for ML projects and the like. The 6700xt is the primary GPU, so if torch is only looking at the primary GPU, that could be my problem, but I will look further into this angle.

As for cl.exe, I can confirm that it is on my PATH. Running cl in command prompt brings up C/C++ Optimizing Compiler, I have version 19.29.30136 x64 if it matters.

I am not 100% sure I understand what exactly you are looking for when you say "Specific Python configuration" I am using python 3.9.7, I use pip and conda to install everything I need, but I'm not certain that's the information you want.

[McLemoreT]

So from what I can tell, picking what GPU torch uses is something you can only do in scripts you write. Since I haven't even gotten to that step (I can't install memtorch), I don't think this is what's wrong. But I'm not an expert, and am interested in what you have to say.

[coreylammie]

Hi @McLemoreT,

In many files within MemTorch, the following statement is used: self.device = torch.device("cpu" if "cpu" in memtorch.__version__ else "cuda") to select the device to be used. Looking at the torch documentation, cuda is equivalent to cuda:X, where X is the result of torch.cuda.current_device().

torch.cuda.set_device() can be used to set the current CUDA device. I have created a separate issue (#114), to add the ability to specify the current active device when compiling/installing MemTorch when CUDA is enabled and multiple GPUs are present. Before merging this, I will spin up a server with two GPUs to verify functionality.

During normal operation, i.e., after MemTorch is installed, torch.cuda.set_device() can be set prior to importing MemTorch. For example, a secondary GPU can be selected to be used as follows:

torch.cuda.set_device(torch.device("cuda:1"))
import memtorch

Update: #115 has been merged to master.

[Philippe-Drolet]

Hello, is CUDA 10.1 required, or would CUDA 11.5 (latest version) also work? Thanks,

I am having issues running memtorch with CUDA on pycharm, here is the log file. I have tried everything #103 suggested to no avail. I am running on windows with cl.exe in my PATH and CUDA-HOME 10.1 also.
log.txt

[coreylammie]

Hi @Philippe-Drolet,

PyTorch, and by extension, MemTorch, do not currently officially support CUDA 11.5. Current progress is being tracked here: pytorch/pytorch#68259. I have personally been able to compile MemTorch with GPU support using CUDA 11.3 (I believe compiling using 11.4 is possible, however).

The same error you are receiving has previously been reported here: https://discuss.pytorch.org/t/how-to-run-detectron2-using-rtx-3090-and-with-cuda-11-x-on-ubuntu/119448/11, i.e., nvcc fatal : Unknown option '-generate-dependencies-with-compile'. This appears to be caused by a mismatch of versions between CUDA and PyTorch.

For MemTorch to be built from source (with CUDA support), the CUDA version associated with the current PyTorch installation and the CUDA Toolkit version must be the same. In addition, the current PyTorch installation must support the CUDA version used to compile MemTorch.

The CUDA version associated with PyTorch can be determined by running the following script:

import torch

if torch.cuda.is_available():
    print(torch.version.cuda)

If possible, I would suggest downgrading CUDA Toolkit to 11.3 or 11.4, until official support is added to PyTorch.
Alternatively, a separate conda environment can be created and used to install MemTorch and its dependancies as follows:

conda create -n memtorch python=3.9
conda activate memtorch
conda install pytorch torchvision torchaudio cudatoolkit=11.3 -c pytorch
pip install memtorch

I hope this helps!

[Philippe-Drolet]

Hello! that will do it! Thanks