Fix attention backend auto-detection for consumer Blackwell (sm120) - #146
Open
tao-plumber wants to merge 1 commit into
Open
Fix attention backend auto-detection for consumer Blackwell (sm120)#146tao-plumber wants to merge 1 commit into
tao-plumber wants to merge 1 commit into
Conversation
is_sm100_supported() used a plain ">=" compute-capability check, so RTX 50-series GPUs (sm120) were misdetected as sm100 datacenter Blackwell and auto-selected the trtllm backend, which only supports Hopper/datacenter-Blackwell trtllm-gen kernels and crashes with "Unsupported architecture" on sm120. Add is_sm120_supported() and route consumer Blackwell straight to the portable fi backend before the sm100 check. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Author
|
@DarkSharpness this may resolve the issue for #129? I've run into same issue on my rtx5080. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Found this issue on my 5080.
Motivition
is_sm100_supported() in python/minisgl/utils/arch.py uses a plain compute_capability >= (10, 0) check to decide GPU support tiers. Consumer Blackwell GPUs (RTX 50-series, sm120) satisfy that check numerically, so they get misdetected as datacenter Blackwell (sm100/B200) and the engine auto-selects the trtllm attention backend. But trtllm's underlying kernels (flashinfer's TensorRT-LLM-Gen FMHA) are only built for Hopper/datacenter-Blackwell — they don't support sm120 at all, so the server crashes immediately on startup with RuntimeError: ... Unsupported architecture in fmhaRunner.cuh. This makes mini-sglang unusable out of the box on any RTX 50-series card with the default --attention-backend auto.
Goal
Make auto backend selection correctly detect consumer Blackwell as its own tier and route it to the portable fi (FlashInfer) backend instead of trtllm, so RTX 50-series users get a working server out of the box without needing to manually pass --attention-backend fi.