[Bug]: DeepSeek-V4-Flash (fused-MoE path) crashes at model load on 74f906b2b — DeepseekV4MoE missing n_logical_experts in extract_moe_parameters

[v1b3coder]

Your current environment

• Branch / commit: ds4-sm120-preview-dev @ 74f906b2b ("test: adapt DS4 prefix cache tests to scheduler block size")
• vLLM version string: 0.1.dev50+g74f906b2b.d20260602
• Base image: vllm/vllm-openai:deepseekv4-cu130 (CUDA 13.0.1), source-built with --no-build-isolation --no-deps
• Hardware: 2× RTX PRO 6000 Blackwell Server Edition (SM120), TORCH_CUDA_ARCH_LIST=12.0, TP=2
• Model: deepseek-ai/DeepSeek-V4-Flash (fp8 checkpoint, quantization=deepseek_v4_fp8)

🐛 Describe the bug

On 74f906b2b, serving DeepSeek-V4-Flash crashes at model load (engine never starts, container restart-loops). Every worker dies with:

AttributeError: 'DeepseekV4MoE' object has no attribute 'n_logical_experts'. Did you mean: 'n_local_experts'?

This happens on the fused-MoE path (_init_fused_moe_experts), which is the default (moe_backend=auto ⇒ use_mega_moe=False) and — as far as I can tell — the only viable path for the fp8 Flash checkpoint, since MegaMoE additionally requires --enable-expert-parallel and fp4 experts (see the guards in DeepseekV4MoE.__init__).

Serve command

vllm serve deepseek-ai/DeepSeek-V4-Flash \
  --tensor-parallel-size=2 \
  --kv-cache-dtype=fp8 \
  --block-size=256 \
  --max-model-len=262000 \
  --max-num-seqs=8 \
  --gpu-memory-utilization=0.96 \
  --max-num-batched-tokens=8192 \
  --enable-prefix-caching --enable-chunked-prefill \
  --no-disable-hybrid-kv-cache-manager --no-enable-flashinfer-autotune \
  --compilation-config='{"cudagraph_mode":"FULL_AND_PIECEWISE","custom_ops":["all"]}' \
  --speculative-config='{"method":"mtp","num_speculative_tokens":2}' \
  --tokenizer-mode=deepseek_v4 --tool-call-parser=deepseek_v4 --reasoning-parser=deepseek_v4 \
  --enable-auto-tool-choice --trust-remote-code

(No --enable-expert-parallel, no custom moe_backend ⇒ fused path.)

Traceback

File "/opt/jasl/vllm/vllm/v1/worker/gpu_model_runner.py", line 5105, in load_model
    self.model = model_loader.load_model(...)
File "/opt/jasl/vllm/vllm/model_executor/model_loader/utils.py", line 61, in initialize_model
    model = model_class(vllm_config=vllm_config, prefix=prefix)
File "/opt/jasl/vllm/vllm/models/deepseek_v4/nvidia/model.py", line 1417, in __init__
    self.set_moe_parameters()
File "/opt/jasl/vllm/vllm/models/deepseek_v4/nvidia/model.py", line 1437, in set_moe_parameters
    self.extract_moe_parameters(example_moe)
File "/opt/jasl/vllm/vllm/models/deepseek_v4/nvidia/model.py", line 1362, in extract_moe_parameters
    self.num_logical_experts = example_moe.n_logical_experts
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1968, in __getattr__
    raise AttributeError(...)
AttributeError: 'DeepseekV4MoE' object has no attribute 'n_logical_experts'. Did you mean: 'n_local_experts'?

Root cause

DeepseekV4MixtureOfExperts.extract_moe_parameters() (vllm/models/deepseek_v4/nvidia/model.py:1362) unconditionally reads six attributes off the example MoE:

self.num_logical_experts        = example_moe.n_logical_experts
self.num_physical_experts       = example_moe.n_physical_experts
self.num_local_physical_experts = example_moe.n_local_physical_experts
self.num_routed_experts         = example_moe.n_routed_experts
self.num_shared_experts         = example_moe.n_shared_experts
self.num_redundant_experts      = example_moe.n_redundant_experts

But only _init_mega_moe_experts() sets those (n_logical_experts, n_physical_experts, n_local_physical_experts, n_redundant_experts, …). _init_fused_moe_experts() sets just n_local_experts / experts_start_idx / experts_end_idx and none of the six. So DeepseekV4MoE.__init__ taking the else: (fused) branch leaves extract_moe_parameters reading attributes that were never assigned.

This looks like a fallout of the MegaMoE / MixtureOfExperts refactor: the new metadata-extraction interface was wired to the MegaMoE init path only.

Regression note

The immediately-preceding state of the branch (caea1cb5, "Add SM120 sparse MLA partial-state prefill") works fine with the exact same command — it has no extract_moe_parameters / n_logical_experts at all, so the fused path loads cleanly. The breakage was introduced with the MixtureOfExperts.extract_moe_parameters addition between caea1cb5 and 74f906b2b.

Suggested fix

Set the expert-count metadata in _init_fused_moe_experts() too (no EPLB redundancy on this path), e.g.:

self.n_routed_experts        = config.n_routed_experts
self.n_shared_experts        = config.n_shared_experts or 0
self.n_redundant_experts     = 0
self.n_logical_experts       = config.n_routed_experts
self.n_physical_experts      = config.n_routed_experts
self.n_local_physical_experts = self.n_local_experts

…or make extract_moe_parameters tolerate the fused path. Happy to test a patch.

[v1b3coder]

The report generated my clanker, but this is human writing. Thank you @jasl for your continuous amazing work. I use your fork intensively and I love it ❤️ .

[jasl]

The latest dev build should fix it (I met the issue in testing too). Could you retry?

[v1b3coder]

Amazing, works for me, too!