Misc. bug: vulkan on 6900xt

[Danik-droid]

Name and Version

Latest vulkan patches cause problems in koboldCPP for radeon 6900xt.

Operating systems

Windows

Which llama.cpp modules do you know to be affected?

llama-quantize

Command line

Problem description & steps to reproduce

I reported a problem with vulkan on the KoboldCPP project page, and was asked to report the problem here (newly released koboldcpp version with latest fixes for vulkan).

LostRuins#1398

As I wrote in the bug report on KoboldCPP, I have two cards GTX 1080ti and Radeon 6900xt. When the creator of Koboldcpp included the latest patches for vulkan, my radeon stopped working properly with vulkan.

I have tested various models (llama 3.1, nemo, mistral small 22/24b) and in none of them vulkan on radeon 6900xt works correctly anymore, either some random characters are generated from the very beginning, or the response loops very quickly and repeats some word.

First Bad Commit

No response

Relevant log output

[0cc4m]

Try to reinstall your driver, the backend is still working fine on my AMD RX 6800 XT on Linux and Windows, so it's likely a problem on your side.

[0cc4m]

Actually, no, there seems to be some issue with the llama 3.1 model you linked, at least. I'll look into it.

[Danik-droid]

Yes, in llama 3 it works ok. But in llama 3.1 models - it generates wrong immediately, or mistral nemo and small 22b/24b not (quickly falls into loops and repeats one word).

I reinstalled AMD drivers (with factory reset) it did not help. Vulkan on 1080ti works correctly.

Vulkan on the previous version of the program works correctly all the time on AMD Radeon 6900xt, even when I run the model on both graphics cards.

[0cc4m]

Yes, thank you for the report. I'll let you know once I found the culprit.

[LostRuins]

thanks for the help

[0cc4m]

@Danik-droid @LostRuins It's hard for me to check, since I don't use Windows and this is Windows-specific, but I'm pretty sure this is the bug already reported in #10710 . It's related to the use of unpack8() in shader code, which is currently bugged in the AMD Windows driver. An AMD employee in the aforementioned issue has already confirmed they fixed it, so it should be working in the next AMD driver version. I expect this to be released within the next few days, as AMD is releasing their new RDNA4 GPUs.