Open
Description
Describe the issue
I have tried running it in both Sd.Next and ComfyUI and both fail when trying to generate an image. There is no error message it just seems to crash the Webui comletely. 4676 and older worked perfectly fine. Since there is no error message I can't really tell you what is broken. I believe the driver team is notified but I'm not sure what they can do since it's not officially supported, so I figured I would also post in here as well.
Wsl2 seems to still work fine.
a750
Windows 11
AOT compiled IPEX for windows
ryzen 5600
32gb of ddr4 at 3200
Metadata
Metadata
Assignees
Labels
Type
Projects
Milestone
Relationships
Development
No branches or pull requests
Activity
jingxu10 commentedon Oct 11, 2023
@min-jean-cho
Nuullll commentedon Oct 11, 2023
+1.
Under some circumstances, I can see "Abort was called at 198 line in file:" -- I believe this is raised from compute runtime.
I'm trying to isolate the issue.
Mindset-Official commentedon Oct 12, 2023
Just to confirm, I also got this a few times.
Vipitis commentedon Oct 12, 2023
accelerate with --use_xpu or ipex enabled in config also throws
exit status 3221225477
with A750 on Windows 10 and driver 4887Nuullll commentedon Oct 15, 2023
It seems that driver 4885 was breaking backward compatibility against previous drivers.
The officially released IPEX Windows JIT wheels work fine with the following reproducer (the image was generated as expected):
However, if I use IPEX AOT wheels built from source with driver 4676 (or earlier) (for example, https://github.com/Nuullll/intel-extension-for-pytorch/releases/tag/v2.0.110%2Bxpu-master%2Bdll-bundle), the program crashes.
trace.log
Crashed while executing
piEnqueueKernelLaunch
for kernel_ZTSZZN2at15AtenIpexTypeXPUL20launch_legacy_kernelIZNS0_18dpcpp_loops_kernelIZZZNS_4impl21copy_device_to_deviceERNS_14TensorIteratorEbENKUlvE3_clEvENKUlvE9_clEvEUlN3c104HalfEE_Lb0ELb1EEEvRNS_18TensorIteratorBaseET_EUliE_EEvxRKSD_ENKUlRN4sycl3_V17handlerEE_clESK_EUlNSI_7nd_itemILi1EEEE_
Probably I should compile IPEX with driver 4885?
Mindset-Official commentedon Oct 15, 2023
You could try and see, but the official wheels haven't been updated (afaik) so I don't think they were compiled on the latest drivers. Maybe the new drivers break something in AOT?
Nuullll commentedon Oct 16, 2023
I tried compiling IPEX AOT for Arc with driver 4887. The reproducer still crashes with the same SYCL PI TRACE log.
Mindset-Official commentedon Oct 26, 2023
Are there any updates on whats going on with the newest drivers? I personally haven't tried the very latest but have heard it is also not working from others.(I may give it a shot if someone says otherwise). Any progress on figuring out what's happening?
Nuullll commentedon Oct 26, 2023
I can confirm that Driver 4885, 4887 and 4900 all cannot work with IPEX AOT, simply because they ship the same Level Zero Compute Runtime "1.3.27193".
Mindset-Official commentedon Oct 26, 2023
I take it this is completely driver level and no way to override and install the older runtime version?
Nuullll commentedon Oct 26, 2023
I tried to replace the driver storage files
ze_intel_gpu64.dll, ze_loader.dll, ze_tracing_layer.dll, ze_validation_layer.dll
underC:\Windows\System32
with the older dlls. But apparently I could've missed something -- failed to load compute runtime library.Mindset-Official commentedon Oct 26, 2023
that's way above my level, however in my folder I do not see a ze_intel_gpu64.dll in the main folder but only in one of the driver state repository folders, this is driver 4676
Nuullll commentedon Oct 26, 2023
Yes, correct. 4
ze_*.dll
in driver storage folder and 3ze_*.dll
in system32. I replaced them all but still got no luck :-(Nuullll commentedon Nov 2, 2023
The issue is gone with Driver 4952