[bitsandbytes.cextension|ERROR]bitsandbytes library load error

[skariapaul]

(unsloth-env) ubuntu@HP-Z2-Mini-G1a:~$ python test_unsloth.py
[bitsandbytes.cextension|ERROR]bitsandbytes library load error: Configured ROCm binary not found at /home/ubuntu/unsloth-env/lib/python3.12/site-packages/bitsandbytes/libbitsandbytes_rocm82.so
Traceback (most recent call last):
File "/home/ubuntu/unsloth-env/lib/python3.12/site-packages/bitsandbytes/cextension.py", line 320, in
lib = get_native_library()
^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/unsloth-env/lib/python3.12/site-packages/bitsandbytes/cextension.py", line 288, in get_native_library
raise RuntimeError(f"Configured {BNB_BACKEND} binary not found at {cuda_binary_path}")
RuntimeError: Configured ROCm binary not found at /home/ubuntu/unsloth-env/lib/python3.12/site-packages/bitsandbytes/libbitsandbytes_rocm82.so
🦥 Unsloth: Will patch your computer to enable 2x faster free finetuning.

[skariapaul]

Finetuning playbook with UnSloth throwing bitsandbytes loading error
https://developer.amd.com/playbooks/unsloth-llms-finetuning/

AMD RYZEN AI MAX+ PRO 395 w/ Radeon 8060S
Ubuntu 24.04.4 LTS

[adamlam2-amd]

Will take a look into this and report back when I can. Thank you!

[lucbruni-amd]

This error is harmless to run through the LoRA training end-to-end, and would be trivialized with bitsandbytes-foundation/bitsandbytes#1878, but that is not included in the pulled-in bitsandbytes version from unsloth yet. However, the "Explore Lower-Memory (4-bit) Fine-Tuning" section is impacted here and I hit the same RuntimeError which blocked my training in the 4-bit path.

The workaround for now would be to build bitsandbytes from source against your system's ROCm so there's no library mismatch:

git clone https://github.com/bitsandbytes-foundation/bitsandbytes.git
cd bitsandbytes
cmake -DCOMPUTE_BACKEND=hip -S .
make -j$(nproc)
# ensure same python venv
pip install .

After that I did not hit any such errors. @adamlam2-amd, how shall we address this in the actual Playbook?

[sreeram-11]

@skariapaul,

Currently, the playbook is performing 16-bit LoRA and does not use bitsandbytes for training. So, that warning/error message can be ignored.

• In the Unsloth Finetuning playook, the training is performed using the PyTorch AdamW optimizer and not a bitsandbytes optimizer.
• bitsandbytes>=0.49.1 is automatically pulled in as a dependency of Unsloth when we run the command:
  pip install "unsloth[amd] @ git+https://github.com/unslothai/unsloth.git"
• Then, when we try to import unsloth, it probes bitsandbytes internally and prints the error when it can't find the matching ROCm binary.

[lucbruni-amd]

@sreeram-11 see my comment above, it does hit the error if you follow the instructions under the "Explore Lower-Memory (4-bit) Fine-Tuning" section.

[danielholanda]

@sreeram-11 Please take a look

[sreeram-11]

@lucbruni-amd, @adamlam2-amd, @danielholanda

As per Luca's suggestion, Explore Lower-Memory (4-bit) Fine-Tuning section has now been removed (PR #290). There are known issues with bitsandbytes ROCm compatibility, so it's best to avoid those instructions for now.