Found by an independent review of the IQ1_S (ggml 19) / IQ1_XXXS (ggml 66) decode that landed in #946, and reproduced here before repair.
Every decode parameter except the grid was pinned only by self-consistency. The grids carry an FNV-1a digest and a lane census, and that seal works. It stops one table short. ReferenceDotF64 (tests/vt/test_ops_quant_dot.cpp:231-236) and the G3 MatmulBTQuant NMSE reference (:915) both decode the weight with vt::cpu::BlockToFloat, which is the function under test, so they are independent only in the summation. Three defects were injected, each applied and compiled, and the whole suite stayed green with an unchanged assertion count:
| Mutation |
Site |
Baseline result |
kIq1sDelta 0.125F to 0.25F (affects BOTH encodings) |
cpu_quant_iq_tables.h:422 |
UNCAUGHT |
| IQ1_S delta sign inverted in dequant AND vec_dot |
cpu_quant_dequant.cpp:389, cpu_quant_dot.cpp:633 |
UNCAUGHT |
IQ1_S scale read from qh bits 13-15 instead of 12-14, both paths |
same two files |
UNCAUGHT |
The NMSE ceiling was widened past the point where it discriminates. tests/vt/test_ops_quant_dot.cpp:148,153 set nmse_max = 2e-3 for both new types. Re-measured over all 12 shapes per type: unmutated iq1_s peaks at 5.240e-4 and iq1_xxxs at 3.109e-4. With the doubled delta iq1_s reads 6.967e-4, so 2e-3 passes the defect and 6e-4 fails it. On iq1_xxxs the same defect moves the statistic the WRONG WAY, 3.109e-4 down to 1.420e-4, so no ceiling catches it there at all: an NMSE against a dequant-f32 reference cannot seal a decode parameter, because both sides decode through the same BlockToFloat.
The loader's expansion path cannot decode the target checkpoint. gguf_dequant.cpp:107-116 lists no case 19 and no case 66, so DequantGgufRowToF32 throws unsupported ggml type although vt::cpu::BlockToFloat decodes both in one line. RouteGgufTensor (gguf_keep_quant.cpp:122) routes a tensor to kExpandBf16 whenever cpu_ref is set (the VT_CPU_REF oracle switch), keep-quant is off, K is ragged, or the role is not verbatim, so the reference lane refuses the model. ggml 18 (IQ3_XXS) has the same omission, pre-existing since the DeepSeek-V4 UD-IQ2_XXS port; it is the ffn_down encoding of that vehicle. All three are one shared switch branch and are fixed together here.
Three smaller ones, all in the same files: cpu_quant_dot.cpp:591 says the IQ1_S kernel reads bsums "and the others do not", while Q2_K, Q5_K and Q4_K already do; cpu_quant_iq_tables.h:1-5 calls the file a 1:1 mirror of llama.cpp @ 237ad9b96 and indexes its tables, but now also carries kIq1xxxsGrid from unslothai/llama.cpp @ 36fe8e1cc and indexes neither new table; and both new census cases in tests/vllm/test_gguf_keep_quant.cpp claim TOTAL coverage of 1702 tensor records while enumerating six of the seven encodings, summing to 864 (F32, 838 tensors, was missing).
Repair. Seal kIq1sDelta against upstream IQ1S_DELTA (ggml-common.h:1121 @ 237ad9b96), and add golden-vector fixtures for all three encodings whose EXPECTED values are produced by the oracles themselves rather than by this tree: ggml_get_type_traits(type)->to_float in a build of ggml-org/llama.cpp @ 237ad9b96 for types 18 and 19, and of unslothai/llama.cpp @ 36fe8e1cc for type 66. The IQ1 inputs are real blk.0.ffn_gate_exps.weight bytes from the two target checkpoints, so this also commits a slice of the 1179648-weight bit-identity run the spec records. Set the NMSE ceiling from the measurement. Complete the census so it sums to the declared record count.
Found by an independent review of the IQ1_S (ggml 19) / IQ1_XXXS (ggml 66) decode that landed in #946, and reproduced here before repair.
Every decode parameter except the grid was pinned only by self-consistency. The grids carry an FNV-1a digest and a lane census, and that seal works. It stops one table short.
ReferenceDotF64(tests/vt/test_ops_quant_dot.cpp:231-236) and the G3MatmulBTQuantNMSE reference (:915) both decode the weight withvt::cpu::BlockToFloat, which is the function under test, so they are independent only in the summation. Three defects were injected, each applied and compiled, and the whole suite stayed green with an unchanged assertion count:kIq1sDelta0.125Fto0.25F(affects BOTH encodings)cpu_quant_iq_tables.h:422cpu_quant_dequant.cpp:389,cpu_quant_dot.cpp:633qhbits 13-15 instead of 12-14, both pathsThe NMSE ceiling was widened past the point where it discriminates.
tests/vt/test_ops_quant_dot.cpp:148,153setnmse_max = 2e-3for both new types. Re-measured over all 12 shapes per type: unmutatediq1_speaks at 5.240e-4 andiq1_xxxsat 3.109e-4. With the doubled deltaiq1_sreads 6.967e-4, so 2e-3 passes the defect and 6e-4 fails it. Oniq1_xxxsthe same defect moves the statistic the WRONG WAY, 3.109e-4 down to 1.420e-4, so no ceiling catches it there at all: an NMSE against a dequant-f32 reference cannot seal a decode parameter, because both sides decode through the sameBlockToFloat.The loader's expansion path cannot decode the target checkpoint.
gguf_dequant.cpp:107-116lists nocase 19and nocase 66, soDequantGgufRowToF32throwsunsupported ggml typealthoughvt::cpu::BlockToFloatdecodes both in one line.RouteGgufTensor(gguf_keep_quant.cpp:122) routes a tensor tokExpandBf16whenevercpu_refis set (theVT_CPU_REForacle switch), keep-quant is off, K is ragged, or the role is not verbatim, so the reference lane refuses the model. ggml 18 (IQ3_XXS) has the same omission, pre-existing since the DeepSeek-V4 UD-IQ2_XXS port; it is theffn_downencoding of that vehicle. All three are one sharedswitchbranch and are fixed together here.Three smaller ones, all in the same files:
cpu_quant_dot.cpp:591says the IQ1_S kernel readsbsums"and the others do not", while Q2_K, Q5_K and Q4_K already do;cpu_quant_iq_tables.h:1-5calls the file a 1:1 mirror of llama.cpp @237ad9b96and indexes its tables, but now also carrieskIq1xxxsGridfromunslothai/llama.cpp @ 36fe8e1ccand indexes neither new table; and both new census cases intests/vllm/test_gguf_keep_quant.cppclaim TOTAL coverage of 1702 tensor records while enumerating six of the seven encodings, summing to 864 (F32, 838 tensors, was missing).Repair. Seal
kIq1sDeltaagainst upstreamIQ1S_DELTA(ggml-common.h:1121@237ad9b96), and add golden-vector fixtures for all three encodings whose EXPECTED values are produced by the oracles themselves rather than by this tree:ggml_get_type_traits(type)->to_floatin a build ofggml-org/llama.cpp @ 237ad9b96for types 18 and 19, and ofunslothai/llama.cpp @ 36fe8e1ccfor type 66. The IQ1 inputs are realblk.0.ffn_gate_exps.weightbytes from the two target checkpoints, so this also commits a slice of the 1179648-weight bit-identity run the spec records. Set the NMSE ceiling from the measurement. Complete the census so it sums to the declared record count.