Torch not compiled with CUDA enabled

[OnlyStopOnTop]

"Torch not compiled with CUDA enabled"

Do get after running python torch2trt.py

Wahts the issue

[tehMArs]

install torch with CUDA matching with your version nvidia-smi, check version
example: pip install torch torchvision --index-url https://download.pytorch.org/whl/cu128

[OnlyStopOnTop]

NVIDIA-SMI 591.86 Driver Version: 591.86 CUDA Version: 13.1

How to install matching torch version?

[tehMArs]

NVIDIA-SMI 591.86 Driver Version: 591.86 CUDA Version: 13.1

How to install matching torch version?

im not sure if its works on newer CUDA driver for such tools, im using 12.8 so you have to downgrade drivers to matches pytorch version 12.8 but if you have rtx 5000 series look for topic here about that.

[OnlyStopOnTop]

can I just install cuda 12.8 and will be downgraded? or do I have to remove driver completly and install older version for nvidia?

[tehMArs]

you can try install cu12.8 with yours drivers, if it fail downgrade it the drivers then

[OnlyStopOnTop]

RuntimeError: Expected tensor for argument #1 'input' to have the same type as tensor for argument #2 'weight'; but type torch.cuda.FloatTensor does not equal torch.cuda.HalfTensor (while checking arguments for cudnn_batch_norm)

[OnlyStopOnTop]

you can try install cu12.8 with yours drivers, if it fail downgrade it the drivers then

is it cuda problem or torch2trt.py file?

[OnlyStopOnTop]

you can try install cu12.8 with yours drivers, if it fail downgrade it the drivers then

Installed cuda 12.8 and visual studio 2022 - still the same problem

Deinstalled all other (new) nvidia drivers and cuda 13.0

Currently cuda version:

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2025 NVIDIA Corporation
Built on Wed_Jan_15_19:38:46_Pacific_Standard_Time_2025
Cuda compilation tools, release 12.8, V12.8.61
Build cuda_12.8.r12.8/compiler.35404655_0

[progmars]

This way it works for me (but without tensorrt):

pip install torch==2.1.0 torchvision==0.16.0 --index-url https://download.pytorch.org/whl/cu121

Pytorch already includes CUDA libraries as indicated in the index url version, so it does not matter what NVIDIA SDK you have installed. However, you need NVIDIA SDK tools for compiling tensorrt and stuff.