Skip to content

Two NVFP4 W4A16 upstream anchors are wrong at the pin, and both point at plausible neighbouring code: __init__.py:879-881 is an FP8 helper's tail (forced-Marlin is 922-924), qwen3.py:271-274 is the class line (packed_modules_mapping is 275-278) #1421

Description

@localai-bot

Two upstream anchors for the NVFP4 W4A16 forced-Marlin selection are wrong at
the parity pin
5559679229bc961848b121ccdeaa8fa5d79bec98. Both were verified
against a local checkout of vllm-project/vllm (origin is upstream, not a
fork) at that exact commit, so this is the pin's own content and not a fork's.

Neither is a defect of any pull request in flight — both predate them — and
both are the shape .agents/porting.md cares about most: the cited line still
exists and still holds plausible, related-looking code
, so a reader who checks
the anchor is confirmed in a wrong belief rather than sent back.

1. kernels/linear/__init__.py:879-881 — off by 43 lines

The text quoted beside the anchor is

elif linear_backend == "auto" and use_a16:
    # Force a16 (Marlin) when running weight-only quantization.
    force_kernel = MarlinNvFp4LinearKernel

At the pin that is vllm/model_executor/kernels/linear/__init__.py
922-924. Line 879-881 is

879        layer_param_names=["weight", "weight_scale", "input_scale", "input_scale_ub"],
880    )
881

which is the tail of init_wfp8_a16_linear_kernel (starts at :848) — an
FP8 helper, not the NVFP4 one. init_nvfp4_linear_kernel does not begin
until :883. The two share the a16 vocabulary, which is exactly why the wrong
anchor reads plausibly.

Cited in code at:

  • include/vllm/model_executor/models/dense_nvfp4_gemm.h:18 (with the quoted line beneath it)
  • include/vllm/model_executor/models/dense_nvfp4_gemm.h:783 (the W4A16 dispatcher's own comment)
  • include/vllm/model_executor/layers/quantization/compressed_tensors/schemes/nvfp4.h:10
  • include/vllm/model_executor/model_loader/gguf_keep_quant.h:201
  • src/vllm/model_executor/models/minimax_h3_device.cpp:153
  • tests/vllm/models/test_qwen3_32b_nvfp4a16_paged_engine.cpp:63
  • tests/vllm/models/test_qwen3_32b_nvfp4a16_load.cpp:19

and in the records at .agents/quantization-matrix.md:124,
.agents/parity-ledger.md:599,915,916, .agents/specs/minimax-h3.md:332,
.agents/specs/sweep-qwen3-32b-nvfp4a16.md:465 and one completed state event.
The related range :842,879-892 appears in the ledger too and needs the same
re-derivation.

2. models/qwen3.py:271-274 — off by 4 lines

packed_modules_mapping is at vllm/model_executor/models/qwen3.py
275-278 at the pin:

275    packed_modules_mapping = {
276        "qkv_proj": ["q_proj", "k_proj", "v_proj"],
277        "gate_up_proj": ["gate_proj", "up_proj"],
278    }

:271-274 is the class Qwen3ForCausalLM(...) declaration itself — adjacent,
so it also reads plausibly.

Cited in code at:

  • include/vllm/model_executor/models/dense_weight_loaders.h:674
  • include/vllm/model_executor/models/dense_nvfp4_gemm.h:627

and in .agents/parity-ledger.md:599 and
.agents/specs/sweep-qwen3-32b-nvfp4a16.md:465.

Why this is not fixed in flow

AGENTS.md fixes a small clear bug in the flow that found it. This is neither
small nor confined: it is nine code and test sites across six files plus six
record files, on paths several rows own, and .agents/parity-ledger.md is one
of the shared record surfaces a broad edit turns into a lock. It also touches
the anchor-checker population, and an anchor checker that reads its expectation
from the file it is checking is a tautology this project has already been bitten
by — so the correction owes a re-derivation of the whole citation set at the pin
rather than a sed.

Found while repairing review findings on
#1415 (A2-Q2b, #810). That pull
request relocates the :783 citation from :725 and deliberately leaves the
value unchanged, so this issue owns every site uniformly rather than half of
them looking reviewed.

What would settle it

Re-derive each citation against 5559679229bc961848b121ccdeaa8fa5d79bec98,
correct the code sites, and decide separately whether historical ledger rows are
corrected or annotated — a ledger row records what was believed on its date, and
rewriting it is a different decision from fixing a live code comment.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions