fix:guard torch.compile in get_smol so first index doesnt crash in windows#30
Open
Qayad-Ali wants to merge 1 commit into
Open
fix:guard torch.compile in get_smol so first index doesnt crash in windows#30Qayad-Ali wants to merge 1 commit into
Qayad-Ali wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hey ,was using gurrt on my machine and came across a bug . it didnt work on my machine bcs I didn't have a C++ compiler installed.Thought i should contribute for this.
What/Issue
ModelManager.get_smol() always calls torch.compile(). On Windows, this can fail because the CPU backend requires MSVC (cl.exe) and CUDA compilation depends on Triton, which isn't supported on Windows.
Impact
Following the README quick-start (/index smolvlm) causes indexing to crash on Windows before any frames are captioned. Reproduced with PyTorch 2.12.1 (CPU).
Fix
Only compile on CUDA-enabled non-Windows systems, and fall back to the eager model if compilation fails. No behavior change for supported Linux/CUDA setups.