[kernels] download kernels when users request for it. - #14298
Conversation
github-actions
Bot
added
documentation
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
| from kernels import get_kernel | ||
|
|
||
| ops = get_kernel("Isotr0py/ggml") | ||
| ops = get_kernel("Isotr0py/ggml", trust_remote_code=DIFFUSERS_TRUST_REMOTE_KERNELS) |
There was a problem hiding this comment.
Wouldn't this break for anyone using kernels<=0.14?
https://github.com/huggingface/diffusers/pull/14298/changes#diff-249c332f10872ba150d634947de12055f99d9db5c3dbf2bef092ccba775f39bdR97
Also trust_remote_code should resolve based on whether downloading remote code is remotely disabled as well.
| ops = get_kernel("Isotr0py/ggml", trust_remote_code=DIFFUSERS_TRUST_REMOTE_KERNELS) | |
| ops = get_kernel("Isotr0py/ggml", trust_remote_code=DIFFUSERS_TRUST_REMOTE_KERNELS and not DIFFUSERS_DISABLE_REMOTE_CODE) |
| from kernels import get_kernel | ||
|
|
||
| ops = get_kernel(_HF_KERNEL_REPO, version=_HF_KERNEL_VERSION, trust_remote_code=True).ops | ||
| ops = get_kernel(_HF_KERNEL_REPO, version=_HF_KERNEL_VERSION, trust_remote_code=DIFFUSERS_TRUST_REMOTE_KERNELS).ops |
There was a problem hiding this comment.
Should also respect global remote code download flag
| ops = get_kernel(_HF_KERNEL_REPO, version=_HF_KERNEL_VERSION, trust_remote_code=DIFFUSERS_TRUST_REMOTE_KERNELS).ops | |
| ops = get_kernel(_HF_KERNEL_REPO, version=_HF_KERNEL_VERSION, trust_remote_code=DIFFUSERS_TRUST_REMOTE_KERNELS and not DIFFUSERS_DISABLE_REMOTE_CODE).ops |
| can_use_cuda_kernels = ( | ||
| os.getenv("DIFFUSERS_GGUF_CUDA_KERNELS", "false").lower() in ["1", "true", "yes"] | ||
| and torch.cuda.is_available() | ||
| and torch.cuda.get_device_capability()[0] >= 7 | ||
| ) |
There was a problem hiding this comment.
Can be done in a follow up. But I think we can remove/deprecate this flag and use DIFFUSER_TRUST_REMOTE_KERNELS as the standard flag for these kinds of operations.
|
@DN6 addressed. |
|
|
||
| ops = get_kernel("Isotr0py/ggml") | ||
| if not DIFFUSERS_TRUST_REMOTE_KERNELS: | ||
| raise ValueError( |
There was a problem hiding this comment.
Think we can just raise a warning here and run without the kernel. It's backwards breaking if we raise an error.
3 participants
We were downloading and loading kernels for
ggufandnunchaku-litewithout asking from users. This PR fixes that.