feat(QUANT-QWEN38-27B-NVFP4-ARM): the second 27B NVFP4 artifact is a ModelOpt checkpoint, and nothing in this tree read its config

[localai-bot]

W5 of #821, and campaign #1574 needs it.

r0b0tlab/Qwen3.8-27B-NVFP4-MTP-sm121 @ 36f717a22990e82c54c1d48ee77c491b87825680 is a second published NVFP4 quant of Qwen3.8-27B, and campaign #1574 needs it. It is not a second revision of the artifact W4 pinned, and it is not the same format.

	unsloth/Qwen3.8-27B-NVFP4 @ 7d6f8d4d (W4)	r0b0tlab/...-MTP-sm121 @ 36f717a2 (W5)
quant_method	compressed-tensors	modelopt
declared by	format: "mixed-precision"	quant_algo: "MIXED_PRECISION"
membership	config_groups, regex targets	quantized_layers, 401 exact module names
ignore	303 entries	empty
FP8 weight scale	per-channel BF16 [out, 1]	per-tensor F32 []
FP8 activations	dynamic per-token, zero input_scale	static, 208 input_scale F32 []
NVFP4 half	W4A4, layers 0-55	W4A16_NVFP4 g16 weight-only, all 64 layers
lm_head	FP8	W4A16_NVFP4
index	1968 names, 2 files	2001 names, 4 files

What this changes

None of W4's four recorded blockers applies to the new artifact. They were properties of the unsloth checkpoint rather than of the format. This one ships the per-tensor static FP8 that LoadFp8Raw already reads and the ModelOpt NVFP4 that LoadNvfp4AnyNaming already reads, so both halves load.

The blocker it exposed instead is that nothing in this tree read its config at all. ct::Config::FromQuantizationConfig stops at quant_method != "compressed-tensors", so W4's whole-checkpoint read answers "" for a ModelOpt checkpoint, and every routing decision falls to the loader's per-projection tensor-NAME probe. That probe cannot be wrong about the bytes and it can be wrong about the checkpoint, in both directions, and both directions load silently: a module declared FP8 that ships an NVFP4 spelling takes the NVFP4 arm, and a module declared unquantized that ships scales takes a quantized arm. Both produce plausible numbers and matching tokens while moving the wrong bytes, which is the defect class AGENTS.md says a token gate cannot see.

LoadQwen3_5Dense now reads the ModelOpt config once for the whole checkpoint, through the shared QuantizationConfigOf lookup, and refuses by name when the declared algorithm and the shipped operand names disagree, when the declared algorithm has no loader here, or when a shipped tensor name belongs to an operand family this resolver has never seen. This is the first production consumer modelopt_mixed_precision.h has ever had; until now its only reachable entry points were its own two unit tests, which is the state W4's record named and did not change.

Why a cross-check and not a router

Routing by the declaration instead of by the probe would move an arm on a gate model. nvidia/Qwen3.6-27B-NVFP4 @ 0893e160 has the same ModelOpt shape and the same 208/193 split; it declares all 193 NVFP4 modules W4A16_NVFP4 and also ships an input_scale on each, which VT_MODELOPT_W4A4=1 reads to select the fp4-activation GEMM. That is a measurement change wearing a correctness change and it needs its own before/after. #1597 owns it, and ## Owed names it.

The cross-check moves no arm: a checkpoint whose config and tensors agree loads exactly as before. Both ModelOpt checkpoints this loader sees were read from the hub before the call site was written, and on both, all 937 weight-bearing modules agree AND all 2001 / 2194 shipped names classify, so both answer "". Every other checkpoint fails IsMixedPrecision at quant_method and reads nothing.

The MXFP8 line in vLLM's loader log is not about this checkpoint

Running the artifact under vLLM prints Detected ModelOpt MXFP8 checkpoint. At the pin 5559679229bc961848b121ccdeaa8fa5d79bec98, ModelOptMixedPrecisionConfig._from_config (modelopt.py:2371-2400) unconditionally constructs all four candidate sub-configs and each warns from its own __init__ (:386, :1035 twice, :1708), so the lines report what was constructed and not what was selected. Selection is ModelOptMxFp8Config.override_quantization_method (:1724-1731), which fires only when the algo string contains MXFP8; per-module dispatch is get_quant_method (:2525-2535), which hands out mxfp8_config only for an entry that says MXFP8. This artifact's 401 entries are 208 FP8 and 193 W4A16_NVFP4, it ships no F8_E8M0 tensor, and its only non-scalar scales are the 193 NVFP4 group scales. No MXFP8 arm is implemented; a declared MXFP8 module is refused by name, and here that refusal never fires. Both halves are asserted so nobody builds an arm on a log line.

Gate

New: tests/vllm/models/test_qwen38_27b_modelopt_mtp_arm.cpp, hermetic, 22 cases / 1687 assertions, over four committed header-only manifests (970 + 976 + 40 + 15 = 2001) and two committed config documents. Every shipped name classifies, nothing is unaccounted in either direction, and the buckets sum to 2001: 720 tensors over 256 modules to FP8, 579 over 193 to W4A16_NVFP4, 702 over 536 unlisted, zero KV-cache scales, zero unclassified. The 256 is separated from the 208 the map names — 48 are linear_attn containers reached by upstream's strategy-3 prefix scan, asserted as 401 kDirect against 48 kPrefix — so no count is right for the wrong reason.

W4's 1968-name accounting is untouched and re-ran green on the same tree at 9 cases / 194 assertions. A gate that accepted both name sets by loosening would have stopped measuring either.

Four shards verified semantically, per AGENTS.md, rather than by a remote hash: header parse plus 8 + header_len + max(data_offsets[1]) equal to the published size for each, and the four sizes exceeding the index's metadata.total_size by exactly the four headers plus their 8-byte prefixes.

One case exists only to say the refusal is SILENT on nvidia/Qwen3.6-27B-NVFP4's shape — wildcard exclude_modules, an input_scale on every NVFP4 module, and a kv_cache_scheme with zero scales — because each of those three could have made it fire on a gate model.

What the fresh review found, and what the second commit repaired

The review mutated each claimed guarantee, and six findings came back. None of the repairs changes what the loader DOES for a checkpoint whose config and tensors agree. What changed is what the gate can SEE.

• Half the guarantee could be deleted green. if (false && ...) on the NVFP4 refusal branch left the suite fully green at 15 cases / 1649 assertions, W4's gate green beside it. That branch is the entire cross-check for the 193 W4A16_NVFP4 modules — 48% of this artifact's 401 and 48% of the sibling's, lm_head included — and nothing asserted it: every case exercised the FP8, the unquantized, the MXFP8 or the unimplemented-algo arm, and the sibling-shape case pins only the direction in which the branch must NOT fire. Two cases now assert the refusal itself, one per shape a checkpoint can ship under that declaration, both entering through LoadQwen3_5Dense. The review's exact mutation now reds both.
• A tensor family the splitter has never seen was skipped, not refused. SplitOperand documents that false means "this resolver has never seen the family, and the caller must NOT read that as unquantized", and Refusal did read it as exactly that: the name belonged to no module, so nothing cross-checked it in either direction. A declared-FP8 module shipping qweight/qzeros/scales LOADED through the production entry point, silently. It is refused by name now, and the blast radius is measured rather than assumed — both artifacts that reach this resolver classify EVERY name they ship, 2001 of 2001 and 2194 of 2194, so the arm cannot fire on either, and a case asserts the first of those.
• The KV-cache skip asserted nothing. Refusal skipped k_scale/v_scale before any module was built and ModuleOperands::Add had no branch for either, so the skip changed no verdict and deleting it left the suite green — while the sibling case carried the comment "the SUFFIXES must stay out of the cross-check whatever a checkpoint does with them", an assertion no assertion could read. The skip is gone and the decision lives in one branch now: Add RECORDS a KV scale and AnyQuantOperand deliberately leaves it out, because kv_cache_quant_algo is a sibling of quantized_layers and a bf16 tower shipping a KV scale must not be refused for "shipping a quantized spelling". Two cases hold the two directions and two mutations red them.
• The manifest count was a hand-maintained literal that Append iterated, so a row deleted without its count was an out-of-bounds read: a SIGSEGV inside case 1 that aborted the binary with 14 cases never run, and a crash is neither a pass nor a fail. The row count is derived with std::size and the literal is asserted against the array rather than trusted by it. The same mutation now reds 5 cases with all 22 having run.
• The test file's header contradicted the case below it, giving the per-scheme composition as 208 / 624, 193 / 579 and 584 / 798 where the assertions read 256 / 720, 193 / 579 and 536 / 702 — a difference of exactly the 48 linear_attn containers and their 96 tensors, which resolve to FP8 through the prefix scan rather than staying unlisted. The asserted set is the measured one and the prose says so.
• The W6 blocker citation named a closed issue. The pinned vLLM oracle BUILDS, installs and imports inside an rc lease on dgx:gpu0, and a model run is still UNTESTED #1185 closed on 2026-08-18 as local-only, three days before the reviewed head, and five sites cited it as the open blocker. W6's NVFP4 token gate cites a CLOSED blocker (#1185); what still blocks it is the DENOMINATOR configuration in a lease, and the 20.4 GiB artifact not being staged #1632 files what actually blocks W6 — the recorded DENOMINATOR configuration surviving a lease, and this artifact's ~20.4 GiB being staged — and the five citations name it. The oracle itself does run a model in a lease; it did so at max_num_batched_tokens 512 against a denominator of 8192, which is a different engine setup rather than a smaller measurement.

Two smaller things ride along. The branch was 24 commits behind origin/main, which is why preflight SKIPPED its commit-trailers and commit-style gates and reported nothing about this tree; it is merged up to origin/main a8f19a5d8 — three merges, because main moved twice during the gate runs — every merge is BUILT, and both gates now run and pass. And the new cases are one per shape rather than grouped under SUBCASE, because a REQUIRE aborts its whole TEST_CASE and a subcase that reds hides every subcase after it — which is how the first red-before run reported one failure where there were three.

Two rows the last merge invalidated

main moved three times while this branch was open, and two of its landings
touched what this one says.

• spec(BENCH-QWEN38-27B-SOTA): the three-way on the competitors' own subject, and three NVFP4 blockers that belong to the artifact rather than to the format (#1574) #1591 (BENCH-QWEN38-27B-SOTA) added a docs/FEATURES.md row for this same artifact reading ☐ UNTRIED. Its key and its facts are kept — 208 input_scale, an F32 scalar weight_scale, "dynamic": false, the three unsloth blockers absent — and only its STATE moves to ◐ LOADS, never RUN, because UNTRIED sitting beside two rows saying the checkpoint loads is a contradiction inside a projection document.
• KV-FP8 W2 and W3: the CUDA fp8 KV store, its paged-attention read, and the runner integration — the only thing now gating every cell of the #1574 three-way #1593 (KV-FP8) landed the CUDA fp8 KV store, which falsifies the comment above IsKvCacheScaleSuffix saying "this tree has no quantized KV cache to apply it to". It now says the narrower thing that is still true: no path in THIS loader reads a checkpoint k_scale/v_scale tensor, and feeding that store from a checkpoint's scales is a different job. Nothing about the refusal changes.

Red before, and the mutations

RED before the call site was wired: 13 cases, 5 failing, every failure refusal was: <empty> — the loader accepted every config/tensor disagreement and both unimplemented algorithms. RED again before the review repair, on the final test file: 22 cases, 3 failing — the AWQ triple and the block-wise scale both answering with an EMPTY refusal, and the KV case reporting they ship weight for a module that also ships k_scale.

Twelve negative mutations on the final tree, every one detected. Each printed its applied git diff --stat, its compile return code and its restore sha256, because a mutation that never applied and a mutation that failed to build both read as a passing test.

Mutation	Cases red
delete the production call site in LoadQwen3_5Dense	8
OperandSuffixes drops .weight_scale_2	7
one manifest row removed, the count literal left at 40	5
StaticFp8() accepts any spelling	3
the container skip removed	3
the NVFP4 refusal branch never fires	2
an unseen operand family is skipped again	2
the unquantized-direction branch never fires	1
the MXFP8 arm carries no reason	1
a KV-cache scale counts as a quantized weight spelling	1
IsKvCacheScaleSuffix stops recognising the two suffixes	1
ModeloptNvfp4() rejects a module that also ships input_scale	1 (the sibling guard, and only it)

Restored tree: 22 cases / 1687 assertions green, and W4's gate 9 cases / 194 assertions green beside it.

Full suite on the same tree: ctest 587 tests, 586 passed, 1 failed, plus three env-gated skips (test_modelopt_mixed_precision_checkpoint, test_voxtral_e2e, test_qwen35_paged_engine). The failure is test_runner, which is RED on main — a block-size case validated on gfx1151/ROCm that a CPU build's attention-backend SELECTOR refuses before CheckKvCacheShape can reach it. It is in a file this change does not touch, filed as #1602 and owned by BACKEND-ROCM. Run on the tree merged up to origin/main 0822160f5, with the whole suite built from scratch (rc 0). The same suite gave the same one failure at the earlier merge point c020347a7 (586 tests), so the verdict is not an artifact of one origin/main.

Not in this change

• Any token or byte measurement. That is W6 and it is blocked on W6's NVFP4 token gate cites a CLOSED blocker (#1185); what still blocks it is the DENOMINATOR configuration in a lease, and the 20.4 GiB artifact not being staged #1632.
• A locally computed sha256. The bytes are ~20.4 GiB and were not mirrored where this unit could reach them, so docs/USAGE.md records the semantic verification and ## Owed records the hash as unpaid. No remote-reported hash was written down as one.
• The FP8 KV arm. hf_quant_config.json asks for kv_cache_quant_algo: "FP8" and the checkpoint ships zero k_scale/v_scale; no production path here reads that file. Refusing it would refuse nvidia/Qwen3.6-27B-NVFP4, which declares the same thing in its config.json and also ships none. Owned by KV-FP8 (KV-FP8 W2 and W3: the CUDA fp8 KV store, its paged-attention read, and the runner integration — the only thing now gating every cell of the #1574 three-way #1593) and named under ## Owed.

scripts/agent-preflight.sh on the final tree: 92 gates ok, 0 SKIPPED, 1 FAIL — test_cpu_x86_llamacpp_floor, the load-dependent red tracked by #618, which reported NO_QUIET_WINDOW on a box carrying two other agent sessions at a one-minute load average above 60. It passed in the ctest run above on the same tree. The branch is merged up to origin/main c020347a7, so commit-trailers and commit-style RAN on this tree for the first time — on the reviewed head they were SKIPPED for being behind and reported nothing at all.

Closes nothing. #821 stays open: its NVFP4 acceptance bullets still need a token gate. #1632 stays open and owns the blocker.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]