Skip to content

[Bugfix] Fix humming kernel crash when layer.has_bias is None#48769

Open
kylesayrs wants to merge 4 commits into
vllm-project:mainfrom
kylesayrs:fix/humming-has-bias-none
Open

[Bugfix] Fix humming kernel crash when layer.has_bias is None#48769
kylesayrs wants to merge 4 commits into
vllm-project:mainfrom
kylesayrs:fix/humming-has-bias-none

Conversation

@kylesayrs

Copy link
Copy Markdown
Contributor

Summary

  • LinearBase.__init__ sets self.has_bias = bias, and some custom model layers (e.g. Hy3-NVFP4-FP8) pass bias=None. This None propagates as has_bias=None into HummingLayerMeta.
  • BaseHummingConfig.to_cpp_str() skips any field whose value fails isinstance(value, (bool, int, Enum))None fails this check, so kHasBias is omitted from the generated C++ struct.
  • NVRTC then fails with: class "LayerConfig" has no member "kHasBias".
  • Fix: convert None → False at the two call sites that forward layer.has_bias to prepare_layer_meta().

Why this is not a duplicate

Searched open PRs for humming has_bias, kHasBias humming, humming NoneType bias — no results.

Test commands and results

Sanity check (basic generation with Hy3-NVFP4-FP8, 4×A100, TP=4):

SANITY CHECK PASSED

GPQA Diamond eval (inspect_evals/gpqa_diamond, 1 epoch, 198 samples):

accuracy  0.778

Notes

  • AI assistance (Claude Sonnet 4.6) was used to identify the root cause and write the fix. All changed lines have been reviewed.
  • The fix is conservative: None → False matches the existing default (has_bias: bool = False in LayerConfig) and is safe for all callers where the layer genuinely has no bias.

🤖 Generated with Claude Code

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@mergify mergify Bot added the bug Something isn't working label Jul 15, 2026

@yewentao256 yewentao256 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for the work!

@yewentao256 yewentao256 added the ready ONLY add when PR is ready to merge/full CI is needed label Jul 15, 2026

@mgoin mgoin left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kylesayrs can we fix the model definition instead? LinearBase's bias arg is typed as bool, so we should try to follow the type (even if it isn't currently being enforced)

def __init__(
self,
input_size: int,
output_size: int,
bias: bool = False,
skip_bias_add: bool = False,
params_dtype: torch.dtype | None = None,
quant_config: QuantizationConfig | None = None,
prefix: str = "",
*,
return_bias: bool = True,
disable_tp: bool = False,
):
super().__init__()
# Keep input parameters
self.input_size = input_size
self.output_size = output_size
self.has_bias = bias

@jinzhen-lin

Copy link
Copy Markdown
Contributor

I feel that the model definition might have been passed incorrectly because the parameter name bias is misleading, it looks like it accepts a bias tensor. Maybe we can change it to something like has_bias?

kylesayrs and others added 4 commits July 18, 2026 13:25
LinearBase sets has_bias = bias, and some custom model layers pass
bias=None. This None propagates into HummingLayerMeta, causing
to_cpp_str() to skip kHasBias (isinstance(None, (bool, int, Enum))
is False), producing a NVRTC compilation error:
  class "LayerConfig" has no member "kHasBias"

Fix both call sites that forward has_bias=layer.has_bias to
prepare_layer_meta() by converting None to False.

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
Some models (e.g. hy_v3) pass bias=None to LinearBase subclasses.
Since has_bias is typed bool, enforce that at the source with bool(bias)
so callers always see a proper bool and no downstream guard is needed.

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
LinearBase.bias is typed as bool, so follow the type at the call site
rather than guarding downstream. Reverts the bool() coercion in
LinearBase.__init__ in favor of the model-level fix.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@kylesayrs
kylesayrs force-pushed the fix/humming-has-bias-none branch from dfb7369 to 241b1d8 Compare July 18, 2026 17:25
@kylesayrs

Copy link
Copy Markdown
Contributor Author

@jinzhen-lin I definitely agree, but there's also precedent from torch.nn.Linear

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working ready ONLY add when PR is ready to merge/full CI is needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants