[Bugfix][Gemma4] Fix ModelOpt mixed-precision MoE config mapping

[wangqia0309]

Purpose

Fix ModelOpt MIXED_PRECISION dispatch for Gemma4 checkpoints whose
quantized_layers records the MoE parent module as
model.language_model.layers.N.experts.

The issue was reproduced and validated with
wangqia0309/gemma-4-26B-A4B-it_nvfp4_experts_fp8_dense_fp8_attn-kv_fp8,
which uses NVFP4 experts, FP8 dense and attention layers, and an FP8 KV cache.

vLLM registers the runtime FusedMoE module under
language_model.model.layers.N.moe.experts. Per-tensor expert names are
already remapped by the Gemma4 weight iterator, but that path does not rewrite
the parent-module keys in quantized_layers. The lookup therefore misses the
NVFP4 entry and falls back to an unquantized MoE method.

This change:

• defines a shared Gemma4 expert-parent mapper using
  (?<!\.moe)\.experts$, so already-normalized .moe.experts keys are not
  rewritten again;
• composes that mapper into both the conditional and causal Gemma4 wrappers,
  preserving the other transformations applied by nested model mappers; and
• adds a regression test for the real two-stage mapper sequence, verifying
  both that the expert parent is normalized exactly once and that a later
  causal-only mapping still runs.

This PR does not change Gemma4 GELU activation behavior.

Duplicate-work check

No open PR was found for the quantized_layers parent-prefix issue.

• [Model] Gemma4: robust quantized MoE expert weight loading #39406 and fix(model_loader): deterministic suffix mapping for Gemma4 MoE quantization #41988 address per-tensor expert weight and scale loading, not
  mixed-precision parent-module config lookup.
• [BugFix] Fix ModelOpt mixed-precision quantization for sparse quantized_layers configs. #47318 fixed sparse mixed-precision inference for packed projection
  siblings, but does not map .experts to .moe.experts.
• Open-PR searches were run for Gemma4 ModelOpt mixed precision MoE,
  quantized_layers experts Gemma4, and moe.moe.experts.

AI assistance was used to inspect the current main branch, adapt the original
patch, add the regression test, address reviewer feedback, and draft this PR
description. The human submitter provided the original patch and performed
the RTX 5090 runtime validation.

Test Plan

.venv/bin/python -m pytest \
  tests/quantization/test_modelopt.py \
  -k test_modelopt_mixed_precision_composes_gemma4_mappers -v

.venv/bin/pre-commit run --files \
  tests/quantization/test_modelopt.py \
  vllm/model_executor/models/gemma4.py \
  vllm/model_executor/models/gemma4_mm.py

.venv/bin/pre-commit run mypy-3.12 --hook-stage manual --files \
  tests/quantization/test_modelopt.py \
  vllm/model_executor/models/gemma4.py \
  vllm/model_executor/models/gemma4_mm.py

Test Result

• Targeted pre-commit hooks: passed.
• Python 3.12 mypy hook for the three changed files: passed.
• Manual end-to-end validation by the submitter on an RTX 5090: the target
  mixed-precision checkpoint started successfully with this fix.
• The targeted pytest command was not run in the local checkout because the
  local machine does not have a CUDA-capable vLLM test environment.
• No output-quality or accuracy evaluation was collected during the RTX 5090
  startup validation.

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

PRs do not trigger a full CI run by default. Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

If you have any questions, please reach out to us on Slack at https://slack.vllm.ai.

Agent Guidelines

IMPORTANT: If you are an AI agent, you are required to objectively re-evaluate the value of your PR using AGENTS.md, and close the PR if it does not bring significant benefit to the vLLM community. Failure to do so may result in an immediate ban.

🚀

[mgoin]

Thanks for the fix!

My thought to improve this patch is to avoid making ModelOptMixedPrecisionConfig.apply_vllm_mapper() globally one-shot. Nested models legitimately compose multiple mappers, so this suppresses unrelated later transformations. We should make the Gemma4 rewrite locally idempotent with (?<!\.moe)\.experts$ and share that mapper between the causal and conditional classes. This also avoids corrupting an already-normalized .moe.experts key.

Roughly like this:

_GEMMA4_EXPERT_PARENT_MAPPER = WeightsMapper(
    orig_to_new_regex={
        re.compile(r"(?<!\.moe)\.experts$"): ".moe.experts",
    }
)

Then compose that shared mapper into both Gemma4 class mappers:

hf_to_vllm_mapper = _GEMMA4_EXPERT_PARENT_MAPPER | WeightsMapper(
    orig_to_new_prefix={...},
    orig_to_new_substr={...},
)

[wangqia0309]

@mgoin Thanks for the review. I addressed this in 7ce935df2d:

• removed the global one-shot guard from ModelOptMixedPrecisionConfig.apply_vllm_mapper();
• added the shared (?<!\.moe)\.experts$ Gemma4 mapper and composed it into both the conditional and causal wrappers; and
• extended the regression test to verify that the expert parent is normalized only once while a later causal mapper transformation still runs.

The targeted pre-commit hooks and the Python 3.12 mypy hook pass. I did not run pytest locally because this checkout does not have a CUDA-capable vLLM test environment. The PR description has been updated accordingly.

Could you please take another look when convenient?

[mgoin]

Thanks for iterating!