Qwen3-Coder gates resolve whatever snapshot happens to be cached: FindQwen3CoderSnapshot takes the first directory with a config.json

[localai-bot]

Three gate files resolve the Qwen3-Coder checkpoint by walking the HF cache and returning the first snapshot directory that contains a config.json:

// tests/vllm/models/test_qwen3coder_paged_engine.cpp:89-101
std::string FindQwen3CoderSnapshot() {
  const fs::path snaps = fs::path(home) /
      ".cache/huggingface/hub/models--Qwen--Qwen3-Coder-30B-A3B-Instruct/snapshots";
  for (const auto& e : fs::directory_iterator(snaps, ec)) {
    if (fs::exists(e.path() / "config.json", ec)) return e.path().string();
  }
  return "";
}

Duplicated verbatim in three files: test_qwen3coder_paged_engine.cpp:89, test_moe_two_engines.cpp:51, test_qwen3_moe_load.cpp:42.

No revision is pinned and none is asserted. If two snapshots are cached, iteration order decides which one the gate measures — and nothing in the output says which.

Why this is the exact weakness hf_snapshot.h exists to close

Sitting beside these, the 27B gate pins its revision explicitly:

// tests/parity/hf_snapshot.h:31-33
// The revision the committed 27B goldens were captured against.
inline constexpr const char* kQwen27NvfP4Revision =
    "890bdef7a42feba6d83b6e17a03315c694112f2a";

That pin was added because a repo was silently re-quantized under the same name — the unsloth 27B advertised as NVFP4 became FP8 without a name change, so every lever measured against it was inert while the gate stayed green. The Coder gates have precisely that exposure today.

The goldens these tests compare against were captured at some revision. Nothing records which, and nothing prevents a future snapshot_download from adding a second directory that silently becomes the subject.

Blast radius

The Coder gate is one of the three SACRED inertness gates (27B / 35B / Coder). It is cited as inertness evidence for every change touching the shared Qwen3.5 loader — including the four arms landed in #490, #740 and #864. Inertness evidence resolved against an unspecified checkpoint is weaker than it reads.

Fix

Pin the revision the goldens were captured against, in hf_snapshot.h alongside the others, and assert the resolved snapshot matches — aborting loudly rather than measuring a different model. parity::Qwen27NvfP4Snapshot() and its siblings are the shape to follow. The three duplicated copies should collapse into that one helper while this is done.

Found while fetching the Coder checkpoint for a SACRED inertness re-run on a reimaged box, where the absence of any cached snapshot made the unpinned resolution visible.

[localai-bot]

A run in progress makes this concrete rather than hypothetical, and sharpens what the defect actually is.

A SACRED inertness re-run on a reimaged box just reported, for the three gates cited as inertness evidence for the shared-loader arms in #490 / #740 / #864:

gate	result
35B	315/315 assertions, M0-EXIT: produced 16/16 tokens — real
27B	Status: SUCCESS, 0 assertions — SKIP, checkpoint absent
Coder	Status: SUCCESS, 0 assertions — SKIP, checkpoint absent

The two skips are not equivalent, and that is the point.

The 27B skip is honest by construction: it is revision-pinned, so an absent 890bdef7a42f… produces a loud, attributable skip.

The Coder skip was honest only by accident. With no snapshot cached, directory_iterator simply had nothing to return. Had any Coder revision been present — a stale one, a partial one, a re-quantized one under the same repo name — that gate would have run against whatever readdir happened to yield and reported a green count for a checkpoint nobody named.

So the failure mode is not "the gate might skip". It is:

a green Coder count is not attributable to any particular checkpoint, and nothing in the output would reveal which one produced it.

That is strictly worse than a skip, because a skip is visible in the assertion count while a mis-attributed pass is not. It is also the same shape as the @ccdaab7e incident that motivated pinning the 27B: the gate stayed green while the subject silently changed underneath it.

Both this observation and the fix direction are unchanged — pin the revision the goldens were captured against, assert the resolved snapshot matches, abort loudly on drift, and collapse the three duplicated copies into that one helper.

[localai-bot]

A third instance of this failure mode, from the same run, and it is the most alarming of the three because the checkpoint was present and hash-verified the whole time.

A SACRED re-run was set up specifically to convert the earlier 0-assertion skips into real counts. Both checkpoints had been fetched and sha256-verified — 26,401,249,207 bytes for the 27B at exactly the pinned 890bdef7a42f…, 61,066,575,656 for Coder. The re-run still reported:

test_qwen27_paged_engine    Status: SUCCESS   0 assertions
test_qwen3coder_paged_engine Status: SUCCESS   0 assertions

Cause: the container ran with HOME=/root and no mount of ~/.cache/huggingface. Both resolvers looked in an empty path, concluded the checkpoint was absent, and skipped — while printing Status: SUCCESS.

So a present, verified, correctly-pinned checkpoint produced exactly the same green output as a missing one. Nothing in the result distinguished:

• checkpoint absent (honest skip)
• checkpoint present but unreachable from this process (environment defect)
• checkpoint present and gate genuinely passed

That is the whole problem in one line. Status: SUCCESS is emitted identically by a gate that proved something and a gate that proved nothing, and only the assertion count separates them — a count nobody reads unless they already suspect the answer.

What this adds to the fix

Pinning the revision (the original ask) is necessary but does not close this. The gates should fail loudly when a checkpoint they were asked to measure cannot be resolved, rather than degrade to a silent skip. A skip is legitimate when nobody staged the model; it is a defect when someone staged it and the harness cannot see it.

Concretely, alongside collapsing the three duplicated FindQwen3CoderSnapshot() copies into a pinned hf_snapshot.h helper:

• have the helper distinguish "not staged" from "staged but unresolvable", and make the second fatal;
• surface the resolved path and revision in the test output, so a green count is attributable on sight rather than by re-derivation.

The environment defect itself was fixed in that run by passing the real HOME, mounting the cache read-only and setting VT_QWEN27_SNAPSHOT — all three checkpoints then logged PRESENT. But the harness should not have needed a human to notice.