Error in dlopen: libcaffe2_nvrtc.so: cannot open shared object file: No such file or directory

[sefgtrdh]

i ran 'heretic Qwen/Qwen3-4B-Instruct-2507' inside a python venv and got
█░█░█▀▀░█▀▄░█▀▀░▀█▀░█░█▀▀ v1.2.0
█▀█░█▀▀░█▀▄░█▀▀░░█░░█░█░░
▀░▀░▀▀▀░▀░▀░▀▀▀░░▀░░▀░▀▀▀ https://github.com/p-e-w/heretic

Detected 1 CUDA device(s):
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /drives/hdd/users/ovvm/heretic/bin/heretic:8 in │
│ │
│ 5 from heretic.main import main │
│ 6 if name == 'main': │
│ 7 │ sys.argv[0] = re.sub(r'(-script.pyw|.exe)?$', '', sys.argv[0]) │
│ ❱ 8 │ sys.exit(main()) │
│ 9 │
│ │
│ /drives/hdd/users/ovvm/heretic/lib/python3.13/site-packages/heretic/main.py:891 in main │
│ │
│ 888 │ install() │
│ 889 │ │
│ 890 │ try: │
│ ❱ 891 │ │ run() │
│ 892 │ except BaseException as error: │
│ 893 │ │ # Transformers appears to handle KeyboardInterrupt (or BaseException) │
│ 894 │ │ # internally in some places, which can re-raise a different error in the handler │
│ │
│ /drives/hdd/users/ovvm/heretic/lib/python3.13/site-packages/heretic/main.py:179 in run │
│ │
│ 176 │ │ count = torch.cuda.device_count() │
│ 177 │ │ print(f"Detected [bold]{count}[/] CUDA device(s):") │
│ 178 │ │ for i in range(count): │
│ ❱ 179 │ │ │ print(f"* GPU {i}: [bold]{torch.cuda.get_device_name(i)}[/]") │
│ 180 │ elif is_xpu_available(): │
│ 181 │ │ count = torch.xpu.device_count() │
│ 182 │ │ print(f"Detected [bold]{count}[/] XPU device(s):") │
│ │
│ /drives/hdd/users/ovvm/heretic/lib/python3.13/site-packages/torch/cuda/init.py:653 in │
│ get_device_name │
│ │
│ 650 │ Returns: │
│ 651 │ │ str: the name of the device │
│ 652 │ """ │
│ ❱ 653 │ return get_device_properties(device).name │
│ 654 │
│ 655 │
│ 656 def get_device_capability(device: Device = None) -> tuple[int, int]: │
│ │
│ /drives/hdd/users/ovvm/heretic/lib/python3.13/site-packages/torch/cuda/init.py:686 in │
│ get_device_properties │
│ │
│ 683 │ Returns: │
│ 684 │ │ _CudaDeviceProperties: the properties of the device │
│ 685 │ """ │
│ ❱ 686 │ _lazy_init() # will define _get_device_properties │
│ 687 │ device = _get_device_index(device, optional=True) │
│ 688 │ if device < 0 or device >= device_count(): │
│ 689 │ │ raise AssertionError("Invalid device id") │
│ │
│ /drives/hdd/users/ovvm/heretic/lib/python3.13/site-packages/torch/cuda/init.py:478 in │
│ _lazy_init │
│ │
│ 475 │ │ │ ) │
│ 476 │ │ # This function throws if there's a driver initialization error, no GPUs │
│ 477 │ │ # are found or any other error occurs │
│ ❱ 478 │ │ torch._C._cuda_init() │
│ 479 │ │ # Some of the queued calls may reentrantly call _lazy_init(); │
│ 480 │ │ # we need to just return without initializing in that case. │
│ 481 │ │ # However, we must not let any other threads in! │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
RuntimeError: Error in dlopen: libcaffe2_nvrtc.so: cannot open shared object file: No such file or directory

i installed pytorch rocm before installing heretic | im using an Rx6700 and running my machine with debian 13
ive also tried running it with the HSA_OVERRIDE_GFX_VERSION="10.3.0" but i got the same result
i tried installing the caffe2 library with pip (since debian doesnt have any official caffe2 libraries) with no luck

pip install caffe2
Collecting caffe2
Using cached caffe2-0.5.0a0.dev100.tar.gz (10.7 MB)
Preparing metadata (setup.py) ... error
error: subprocess-exited-with-error

× python setup.py egg_info did not run successfully.
│ exit code: 1
╰─> [15 lines of output]
Traceback (most recent call last):
File "", line 2, in
exec(compile('''
~~~~^^^^^^^^^^^^
# This is -- a caller that pip uses to run setup.py
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
...<32 lines>...
exec(compile(setup_py_code, filename, "exec"))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
''' % ('/tmp/pip-install-gdnndbjh/caffe2_324bea98091840278a494f003b95c01f/setup.py',), "", "exec"))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "", line 35, in
File "/tmp/pip-install-gdnndbjh/caffe2_324bea98091840278a494f003b95c01f/setup.py", line 108, in
from tools.setup_helpers.env import check_env_flag, check_negative_env_flag
ModuleNotFoundError: No module named 'tools'
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

[p-e-w]

This is pretty clearly a problem with your accelerator stack (CUDA/driver/PyTorch). Try some other software that uses Transformers, you will very likely see the same exception.