[memory] Trellis/W4A16 scratch sizing: unpinned block-size sweep reserves +65 MiB/rank worst case; plan scales unbounded with max_tokens

[malaiwah]

Provenance: found during a memory review of the v20 r12 release image voipmonitor/vllm:gilded-gnosis-v20-vllmb46c3aa-si35aebc6-fi801d57a-cu132-20260730-r12; code refs are against the composed r12 tree, whose sparkinfer HEAD is de04125 (reachable on origin/fix/trellis-stable-profile-20260730 / pr/92), so blob links below point at that commit. Deployment used for impact math: GLM-5.2 753B MoE, EXL3 3.0bpw, 4x RTX 6000 Pro Blackwell (sm120, 188 SMs), TP4 rank-sliced Trellis MoE: k=6144, n=512 per rank (fc1_cols=1024), E=160, topk=8. These are proposals from a source review, not demands — happy to be corrected on intent.

1. Unpinned w4a16_block_size_m makes plan sizing sweep all block sizes and reserve the worst case (+~65 MiB/rank at a 64-token plan)

Where: sparkinfer/moe/fused_moe/_impl.py _plan_core_workspace, L2577-L2608:

        block_sizes = (
            (int(w4a16_block_size_m),)
            if w4a16_block_size_m is not None
            else _W4A16_ALLOWED_ROUTED_SIZES
        )
        for block_size in block_sizes:
            route_slots = max_packed_route_slots(
                routed_capacity,
                int(block_size),
                route_E,
            )
            ...
            fc1_c_tmp_elements = max(
                fc1_c_tmp_elements,
                packed_gemm_scratch_elements(
                    size_n=fc1_cols, route_slots=route_slots,
                    moe_block_size=int(block_size), sms=sms,
                ),
            )
            fc2_c_tmp_elements = max(...)

with _W4A16_ALLOWED_ROUTED_SIZES = (8, 16, 32, 48, 64) (host.py L16) and max_packed_route_slots(numel, b, E) = numel + E*(b-1) for numel >= E (host.py L222-L229).

What/Why: TPMoEScratchCaps.w4a16_block_size_m defaults to None (_impl.py L754), and plan_tp_moe_scratch / plan_tp_moe_arena_layout forward it verbatim (L6390, L6425, L6829). Any caller that does not pin the block size gets both c_tmp planes (and packed_route_indices/block_expert_ids) sized as the max over all five launch variants, dominated by block 64's padded route slots — capacity reserved for launch shapes a decode-window plan never picks.

Impact (recomputed, 64-token plan, routed_capacity=512, E=160, sms=188, fc1_cols=1024, k=6144):

block	route_slots	fc1_c_tmp	fc2_c_tmp
8 (pinned)	512+160·7 = 1,632	11.75 MiB	11.75 MiB
64 (sweep max)	512+160·63 = 10,592	min(1024·10592, 188·4·64·256)·4 B = 41.38 MiB	min(6144·10592, 12,320,768)·4 B = 47.0 MiB

Pinned block 8: 23.5 MiB total; unpinned sweep: 88.375 MiB → +64.9 MiB per rank per plan. Against the production turnkey margin of ~4.1 GiB/GPU this is ~1.6% of the entire margin per unpinned plan, i.e. ≈4,600 KV tokens at DCP2/nvfp4 (100 MiB ≈ 7,100 tokens).

Reachability (honest): the shipped vLLM integration does not hit this — exl3.py _rank_sliced_runtime always passes w4a16_block_size_m (_positive_env_int("VLLM_EXL3_TRELLIS_BLOCK_M", 8) for the decode plan, VLLM_EXL3_PREFILL_BLOCK_M default 64 for the prefill plan), and the MTP draft layers share the same runtime constructor, so both plans are pinned even with the env vars unset. The exposure is any direct api.Caps(...) consumer (benchmarks, ad-hoc runs, future callers) that leaves the field at its None default: they silently pay the worst-case reserve.

Suggested fix: derive the sweep set from select_route_block_size_m over the planned token counts (or over {1..max_tokens} extremes) instead of the full allowed tuple when w4a16_block_size_m is None, so the reserve matches block sizes the launcher can actually select for that capacity.

Verification: on the r12 tree, build two TPMoEScratchCaps for the same trellis weight plan (max_tokens=64, num_topk=8) with w4a16_block_size_m=8 vs unset, call plan_tp_moe_scratch(caps).scratch_specs() and diff the arena byte totals; the delta is the ~65 MiB above (fc1/fc2 c_tmp fp32 tensor specs at _impl.py L2652-L2653).

2. plan_tp_moe_scratch scales unbounded with caps.max_tokens — no clamp/guard against Trellis capacity (~1.0 GiB/rank at 3072 tokens, ~2.6 GiB at 8192)

Where: sparkinfer/moe/fused_moe/_impl.py _plan_core_workspace full-rotation staging, L2641-L2683:

        tensor_specs = [
            _TensorAllocSpec(
                "intermediate_cache13",
                (routed_capacity * max(fc1_cols, int(k)),),
                cache_dtype,
            ),
            ...
        ]
        if full_rotation:
            max_tokens = max(routed_capacity // max(int(num_topk), 1), 1)
            tensor_specs.extend(
                (
                    ...
                    _TensorAllocSpec("full_rotation_output", (max_tokens, int(k)), torch.float32),
                    _TensorAllocSpec("rotation_a_gate", (routed_capacity, int(k)), torch.float16),
                    _TensorAllocSpec("rotation_a_up", (routed_capacity, int(k)), torch.float16),
                    ...
                )
            )

routed_capacity = max_tokens × num_topk (via plan_tp_moe_execution, routed_rows = num_tokens * num_topk, L5365), and neither plan_tp_moe_scratch (L6368) nor TPMoEScratchCaps.__post_init__ (which only floors max_tokens at 1, L759) applies any upper clamp, capacity assertion, or even a log line.

What/Why: every staging tensor scales linearly in max_tokens × topk × k at fp16 (rotation_a_gate, rotation_a_up, intermediate_cache13) plus fp32 full_rotation_output at max_tokens × k. A single mis-sized caps.max_tokens (env typo in VLLM_EXL3_TRELLIS_MAX_M, a runtime-cache-key miss, a direct API caller passing a batch bound instead of a decode window) silently allocates a multi-GiB arena with no diagnostic.

Impact (recomputed, k=6144, topk=8, E=160, sms=188, block 64):

max_tokens	cache13	rotation_a_gate+up	full_rotation_output	c_tmp (fc1+fc2)	plan total
32 (decode plan, block 8)	3.0 MiB	6.0 MiB	0.75 MiB	22.5 MiB	≈32.5 MiB
3072	288 MiB	576 MiB	72 MiB	94 MiB	≈1,054 MiB/rank
8192	768 MiB	1,536 MiB	192 MiB	94 MiB	≈2,654 MiB/rank

Reachability (honest): this is not merely hypothetical. The shipped vLLM integration (exl3.py _rank_sliced_runtime) deliberately plans a second prefill Trellis arena at plan_max_tokens = max_num_batched_tokens whenever VLLM_EXL3_PREFILL_TRELLIS (default on) and max_batched_tokens > max_trellis_m — so at the production turnkey (max_num_batched_tokens=3072) the ~1.0 GiB/rank figure above is an intended allocation, and at the 8192-token release-gate config it is ~2.6 GiB/rank. That is by design on the caller side; the finding here is that sparkinfer's plan API gives the caller no guardrail. Notably, the vLLM code itself carries a comment documenting a past incident of exactly this failure mode ("any m above the planned capacity produced a cache MISS — silently planning a fresh ~1 GiB arena mid-serve"). On a deployment with ~4.1 GiB total margin/GPU, one accidental oversized plan is fatal (OOM) rather than a graceful error.

Suggested fix: in plan_tp_moe_scratch (or TPMoEScratchCaps.__post_init__), validate max_tokens against an explicit caller-supplied ceiling (e.g. a max_plan_tokens cap in the weight plan or an opt-in allow_large_plan flag), or at minimum emit a warning with the computed arena bytes when the full-rotation plan exceeds a threshold (say 256 MiB), so an accidental 3072/8192-token plan is loud instead of silent.

Verification: on the r12 tree, plan_tp_moe_scratch(Caps(max_tokens=3072, num_topk=8, quant_mode="w4a16", weight_plan=<trellis3_t256 plan with k=6144, n=512, E=160>, w4a16_block_size_m=64)).scratch_specs() — summing the spec byte sizes reproduces the ≈1,054 MiB above; no warning or error is raised at any max_tokens.

---

Also examined and not reported: the if moe_block_size == 8: elements *= 2 term in packed_gemm_scratch_elements (host.py L264-L270) initially looked like an over-ask applied after the min, but it matches real kernel addressing — block-8 kernels reserve a full 16-row m-block region per split-K lock slot (cta_m_blocks = _covering_count(8, 16) = 1; c_size_int4 = cta_m_blocks*16*cta_n_blocks*16 // 4, kernel.py L2479-L2480) and the m8 merge walks even slot indices {0,2,4,6} with stride active_threads across that region (kernel.py L2483-L2502, L2643), so the true full-materialization bound is 2 × size_n × route_slots and the doubled min is exact, not an overshoot.

[malaiwah]

Independent re-review by a second reviewer against the same trees (sparkinfer 35aebc6 as composed into the r12 image — line numbers match the de04125 citations — plus vLLM b46c3aa); every number below was recomputed from scratch, retractions and corrections included. The headline max_tokens question in §2 was the claim I tried hardest to break.

Claim	Verdict	Note
§1 mechanism: unpinned w4a16_block_size_m sizes c_tmp/route buffers as the max over all 5 allowed block sizes	HOLDS	_impl.py L2577-2608 re-read; sweep is over _W4A16_ALLOWED_ROUTED_SIZES, max() per plane
§1 arithmetic: +64.9 MiB at a 64-token plan	HOLDS	reproduced byte-exact: 64.91 MiB (23.50 → 88.38 MiB of c_tmp; full arena 43.51 → 108.42 MiB)
§1 reachability: vLLM always pins; exposure is direct Caps consumers	HOLDS-WITH-CORRECTION	narrower and wider than filed — see §1a. For trellis3_t256, unpinned is a hard error at plan time, so a Trellis Caps consumer can never observe the reserve; but ep_moe sweeps unconditionally with no pin at all
§1 verification recipe as written	HOLDS-WITH-CORRECTION	the recipe raises RuntimeError on the unset case for a Trellis weight plan; use plan_tp_moe_arena_layout or a non-full-rotation w4a16 plan (§1a)
§2 mechanism: plan scales linearly in max_tokens × topk × k, no clamp/assert/log	HOLDS	L2641-2683 re-read; __post_init__ (L758-759) only floors at 1
§2 the load-bearing fact: the prefill plan receives max_num_batched_tokens, not VLLM_EXL3_PREFILL_CHUNK	HOLDS	traced end to end in §2a. VLLM_EXL3_PREFILL_CHUNK=128 never reaches Caps.max_tokens
§2 arithmetic: ≈1054 MiB @3072, ≈2654 MiB @8192	HOLDS	reproduced to the byte: 1054.14 / 2654.30 MiB. Independently corroborated by your own source comment — see §2b
§2 reachability: ~1.0 GiB/rank is allocated at turnkey 3072	HOLDS	one arena per (model scope, device), not per layer; eager-only, never inside a captured graph (§2c)
§2 decode-plan row "32 (decode plan)"	HOLDS-WITH-CORRECTION	the turnkey pins VLLM_EXL3_TRELLIS_MAX_M=64, so the shipped decode plan is 64 tokens / 43.51 MiB, not 32 / 32.5 MiB. (32.5 MiB is exactly right for a 32-token plan.)
§2 "sparkinfer's plan API gives the caller no guardrail"	HOLDS-WITH-CORRECTION	no automatic guardrail, agreed — but a byte-exact pre-flight does exist; it is just private and not cheap for Trellis (§2d)
Closing note: the moe_block_size == 8 → elements *= 2 term is exact, not an over-ask	HOLDS	re-derived independently; agree it is not a finding

---

§1a Reachability, corrected

Two corrections in opposite directions.

Narrower for Trellis. plan_tp_moe_scratch unconditionally calls _plan_full_rotation_w4a16_launches, which refuses an unpinned plan (_impl.py L6199-6202):

    capacity_tokens = max(int(capacity_tokens), 1)
    if not core_plan.route_block_size_m:
        raise RuntimeError(
            "Trellis launch planning requires the arena's route_block_size_m"
        )

So on a trellis3_t256 / exl3_trellis_mcg plan — the deployment class in the issue header — leaving the field at its None default does not silently reserve +65 MiB; it fails to produce a plan at all. The +64.9 MiB is therefore reachable only via (a) plan_tp_moe_scratch on a non-full-rotation w4a16 layout (packed / nf3_2p1), or (b) plan_tp_moe_arena_layout directly, which computes total_nbytes without going through the launch planner. This also means the §1 verification recipe as written raises on the unset case; it needs a non-Trellis weight plan (or the layout API) to show the delta.

Wider elsewhere. ep_moe/_impl.py L382-408 runs the identical worst-case sweep with no pin parameter at all — EPMoEScratchCaps has no w4a16_block_size_m — so every EP plan pays the full reserve unconditionally. If the sweep is worth narrowing in fused_moe, that planner needs the same treatment or it becomes the only remaining offender.

The over-reserve is provable, not just probable. Runtime resolution when unpinned is select_route_block_size_m(token_count, num_topk, route_E) (_impl.py L6542-6544), and _ROUTED_SIZE_TARGET_FILL = 0.9. At topk=8 / E=160, avg_routes_per_expert = m/20, so block 8 is selected for all m < 144. For a 64-token plan the reachable block set is exactly {8} — blocks 16/32/48/64 are unreachable at that capacity by construction, and the pinned-8 size is the correct size. That makes the suggested fix mathematically tight rather than heuristic.

§2a The max_tokens trace (the check that decides this issue)

VLLM_EXL3_PREFILL_CHUNK=128 is set in the shipped turnkey env, so the obvious way to kill this claim is if the prefill plan were chunk-sized. It is not. vllm/model_executor/layers/quantization/exl3.py:

1193:            layer.exl3_max_num_batched_tokens = int(
1194:                scheduler_config.max_num_batched_tokens
1195:            )
...
1535:        max_batched_tokens = int(layer.exl3_max_num_batched_tokens)
1536:        prefill_plan_enabled = prefill_trellis and max_batched_tokens > max_trellis_m
...
1611:        if prefill_plan_enabled:
1612:            prefill_plan, prefill_scratch = _plan_with_scratch(
1613:                max_batched_tokens, prefill_block_m
1614:            )

and _plan_with_scratch(plan_max_tokens, plan_block_m) (L1582-1602) puts that value straight into api.Caps(max_tokens=plan_max_tokens, …). chunk is used only for the eager parity staging — parity_rows = min(chunk, max_batched_tokens) (L1537-1541) and the xh/out32/tg/tu/ig/iu/flat_token/ones buffers (L1647-1706). It never touches Caps.

Both ways, at GLM shapes (k=6144, topk=8, E=160, sms=188, block 64, fp16 cache):

Caps.max_tokens	arena
3072 (max_num_batched_tokens, actual)	1054.14 MiB
128 (VLLM_EXL3_PREFILL_CHUNK, hypothetical)	130.42 MiB

Worth noting even the counterfactual is not small: fc2_c_tmp saturates its sms*4*block*256 cap at 47.0 MiB independently of token count, so a chunk-sized plan would be ~130 MiB, not the ~36 MiB a purely linear extrapolation suggests. Moot here, but it is the reason a "just plan the chunk" fix has a floor.

Turnkey config confirmed from the container: --max-num-batched-tokens 3072, VLLM_EXL3_TRELLIS_MAX_M=64, VLLM_EXL3_PREFILL_BLOCK_M=64, VLLM_EXL3_PREFILL_TRELLIS unset (defaults on). 3072 > 64 ⇒ the second plan is built.

§2b Arithmetic, independently reproduced — and cross-checked against your own comment

Recomputing every spec in L2641-2683 with 16-byte inter-tensor alignment (_impl.py L2996-2998):

component	3072	8192
intermediate_cache13 (rc × max(fc1_cols,k) fp16)	288.00	768.00
rotation_a_gate + rotation_a_up (rc × k fp16)	576.00	1536.00
intermediate_cache2 (rc × n fp16)	24.00	64.00
full_rotation_output (max_tokens × k fp32)	72.00	192.00
fc1_c_tmp + fc2_c_tmp (both cap-limited at block 64)	94.00	94.00
route/expert int32 + kernel_workspace	0.14	0.30
total	1054.14 MiB	2654.30 MiB

Matches the filed figures exactly. The model is also validated against a number you wrote yourselves: plan_tp_moe_arena_layout disables power-of-two token bucketing for EXL3 precisely because of this arena (L6088-6093):

        # Trellis plans one fixed caller-owned capacity and its route-pack
        # wrapper already falls back to that exact capacity when the supplied
        # buffers are smaller than the generic compile bucket. Rounding a
        # 3072-token EXL3 plan to 4096 therefore reserves roughly 320 MiB of
        # unreachable activation scratch without reducing launch variants.
        bucket_w4a16_tokens=source_format != "exl3_trellis_mcg",

My model gives arena(4096) − arena(3072) = 1374.17 − 1054.14 = 320.03 MiB — i.e. your "roughly 320 MiB" falls out of the same byte formula that produces the 1054 MiB headline. (Also: had exl3_trellis_mcg not been excluded from bucketing, the turnkey arena would be 1374 MiB, so this exclusion is already worth ~320 MiB/rank in the shipped image — good call.)

Only intermediate_cache2 (24 MiB) depends on the per-rank n=512 assumption; fc1_c_tmp stays cap-limited even at n=384, so the headline is insensitive to that shape (≤2.3% of the total).

§2c Reachability details worth stating precisely

• Per rank, not per layer. _RANK_SLICED_RUNTIMES is keyed by (_runtime_owner_token, device_index, dtype, hidden, intermediate_per_partition, local_num_experts, topk, max_batched_tokens, min/max_trellis_m, block_m, chunk, prefill_trellis, prefill_block_m, tile_config) (exl3.py L1551-1570) — no layer identity — so all same-shape GLM MoE layers share one runtime. 1054 MiB/rank total, not ×78.
• Turnkey per-rank Trellis total = decode plan (64 tok, block 8) 43.51 MiB + prefill plan 1054.14 MiB = 1097.65 MiB/rank ≈ 78k KV tokens at DCP2/nvfp4 (7,100 tok per 100 MiB). §1's 64.9 MiB ≈ 4,600 tokens — that conversion in the issue is right.
• No per-step cost, and never inside a captured graph. Planning is refused under capture (exl3.py L1574-1578), and the prefill plan is only used for m > max_trellis_m = 64 while the turnkey captures 4..64 — so the prefill path is exclusively eager. This is pure steady-state VRAM, not an amortizable per-step cost. Conversely: it is allocated during the profiling forward, so it is inside vLLM's memory profile and does correctly reduce the KV budget rather than threatening OOM.
• Not doubled at turnkey. The draft-side duplicate (vllm#203) needs the draft on the EXL3 backend; the turnkey draft runs moe_backend=triton, so the ×2 is release-gate-only.

§2d The guardrail that does exist

plan() returns the byte-exact requirement without allocating the arena — the caller allocates from scratch_specs()[0] — so a caller can price a capacity before committing. Two caveats that keep the claim substantially intact: for full_rotation plans plan_tp_moe_scratch also JIT-compiles the launch family (L6434 → L6229+), so it is not a cheap budgeting query; and the cheap one, plan_tp_moe_arena_layout, is private (_impl, absent from fused_moe.META.entry_points). Suggested reframing of §2's ask: export a sizing query (plan_tp_moe_arena_layout, or Caps.arena_nbytes()) rather than add a policy clamp.

---

Fix-safety notes

§1 — derive the sweep set from select_route_block_size_m instead of the full tuple. Capture-safe: plan-time only, changes no allocation behaviour at bind/run. Numerically inert: the block size the launcher picks is unchanged; only the reserve shrinks. Three invariants:

1. The derived set MUST use the same route_E the runtime uses (workspace.route_E at L6542), not weight_E. An EP-style caller passing a smaller route_num_experts raises avg_routes_per_expert and makes larger blocks selectable — deriving with the wrong E under-reserves.
2. It MUST cover every live m ≤ planned capacity, not just m == max_tokens. select_route_block_size_m is monotone non-decreasing in m and packed_gemm_scratch_elements is monotone in block size, so {select(max_tokens, topk, route_E)} alone dominates the whole range — but that is a property to assert in a test, not to leave implicit.
3. packed_route_indices / block_expert_ids shrink with the same sweep and are handed to the kernel directly. c_tmp under-reserve is fail-safe — _get_c_tmp (w4a16/kernel.py L9164-9184) checks scratch.numel() >= elements and falls back to torch.empty, raising cleanly under capture — but the route buffers have no such fallback, so size them from the derived set too, not from a hardcoded assumption.

To confirm: diff plan_tp_moe_arena_layout(..., w4a16_block_size_m=None).total_nbytes against =8 on a packed-layout w4a16 plan at these shapes; expect 64.91 MiB. Nothing to benchmark — this is a pure reserve change.

§2 — DO NOT clamp, and do not assert against a library-chosen ceiling. This is the part of the filed suggestion I would push back on hardest.

• A hard assert/ceiling would reject the shipped vLLM prefill plan (3072 turnkey, 8192 release gate). Both are working, validated configs; the arena is intentional on the caller side. A library-side ceiling turns a passing release gate into a startup failure.
• A silent clamp is loud but fatal in the worst place. The run path does guard (_impl.py L9988-9991, "W4A16 TP MoE binding capacity is too small"), and vLLM's own guard (exl3.py L1753) compares m against its own max_batched_tokens rather than the plan's realized capacity — so a clamped plan passes startup and profiling and then raises on the first full-size prefill chunk mid-serve. Strictly worse than today.
• The 256 MiB warning threshold as specified would fire on every shipped launch (the gate arena is 2.65 GiB by design), and the caller already logs the exact number: "EXL3 rank-sliced runtime planned: … prefill trellis block_m=%d arena=%.1fMiB" (exl3.py L1714-1728). So the marginal diagnostic value is near zero — which is why I'd swap the warning for the exported sizing query in §2d.

The actual GiB lever is caller-side and does need measurement. Nothing in sparkinfer forces the plan capacity to equal max_num_batched_tokens; vLLM could plan a smaller capacity and loop the MoE call over row slices. From the same byte model:

plan capacity	arena	launches/layer per 3072-token chunk	routed rows/launch
3072 (today)	1054.1 MiB	1	24,576
1536	574.1 MiB	2	12,288
512	254.1 MiB	6	4,096

512 rows would free ~800 MiB/rank ≈ +57k KV tokens/GPU at DCP2/nvfp4, at 6 launches/layer/chunk still carrying 4,096 routed rows each. Whether that is free depends on per-launch prologue and expert-tile re-streaming across slices at block 64 — measure prefill tok/s and TTFT on a 3072-token chunk at capacities {3072, 1536, 1024, 512} before believing it. That belongs on the vLLM side (adjacent to vllm#203); the sparkinfer-side prerequisite is only the sizing query so the caller can price the tradeoff.

Priority triage

• Trivial + safe: narrow the §1 sweep to the reachable block set (library-default robustness; the shipped integration is unaffected either way), and mirror it into ep_moe/_impl.py L382 which has no pin escape at all. Export plan_tp_moe_arena_layout (or add Caps.arena_nbytes()).
• Needs measurement: the ~0.8 GiB/rank prefill-capacity chunking — real lever, caller-side, throughput risk unquantified.
• Drop: the 256 MiB warning threshold and any max_tokens clamp/assert as specified — the first is redundant with the caller's existing log line, the second would reject two known-good shipped configs.

[malaiwah]

Implementation is now open as stacked PR #100: #100

It covers the safe work from this issue:

• narrows unpinned fused/EP W4A16 scratch to the conservative runtime-reachable block interval;
• exports fused_moe.required_nbytes(Caps) without launch compilation or retained storage;
• does not add a clamp/warning or implement caller-side prefill chunking.

Observed at the filed 64-token/topk8/E160/SM188/6144x512 non-full shape: 94.94 -> 30.03 MiB, 64.91 MiB/rank removed. The public query prices the 3072-token Trellis configuration at 1055.36 MiB including the complete arena.

Independent adversarial review found two hazards in the first narrowing and both are fixed with exact regressions: direct-topk/TC-decode c_tmp geometry, and non-Trellis core routing against weight_E when router route_num_experts is smaller. Final review verdict: APPROVE. Final CPU-isolated affected suites: 32 pass / 80 GPU skip; Ruff passes. The current production deployment remains on its original image and healthy.

[malaiwah]

Caller-side completion is now open as vLLM PR #210: local-inference-lab/vllm#210

Together with SparkInfer PR #100, this closes the surviving actionable scope:

• moe: bound W4A16 scratch and repair tail correctness #100 narrows runtime-reachable block planning and exports the cheap byte-exact sizing query.
• #210 adds opt-in VLLM_EXL3_PREFILL_CAPACITY; unset/blank remains exactly max_num_batched_tokens.
• Explicit malformed, non-positive, and above-scheduler values fail before planning; nothing is clamped.
• Larger eager prefills are dispatched as contiguous capacity-sized slices plus an exact short tail through one plan/scratch, preserving matching input, route-weight, route-ID, and output views in homogeneous and mixed paths.
• Runtime cache identity and startup arena/buffer logging include the selected capacity.

CPU-isolated vLLM dispatch-contract suite: 12 passed; changed-file Ruff passes. The current 3072-token GLM plan is about 1.05 GiB/rank; a 512-row opt-in plan prices near 254 MiB/rank. No throughput default is asserted—the default stays unchanged until live TTFT/prefill measurement justifies a deployment choice. All GPUs remain occupied by the unchanged production service, so no live performance result is claimed.

[malaiwah]

Rental validation initially reproduced the planner saving but found an ordered #100 + #102 scratch-reuse failure. Promotion was correctly blocked while that failure was investigated.

The repair campaign is now complete:

• current moe: bound W4A16 scratch and repair tail correctness #100 head c2f135c66032ac5e9d0778067dd19ae1910cff47;
• exact moe: bound W4A16 scratch and repair tail correctness #100 + fix(moe): reject unresolved capture launches #102 composition cae3aad137bd739881833c064e1912177698156f;
• planner saving verified at 64.90904 MiB/rank;
• contract/poisoned-replay/bounds matrix: 88/88;
• Compute Sanitizer: 24/24, 0 errors;
• composed union: 220 passed, 16 skipped cold and warm;
• matched three-run A/B: byte-identical output/allocation/peak-memory, latency delta within -0.084% to +0.068%.

Four pre-existing base-kernel correctness/memory-safety defects were repaired; none was introduced by the planner commit. The smaller arena exposed the stale-lane failure, while the expanded boundary/sanitizer campaign found the other three. Their consequences included wrong/non-finite output and 278 invalid global reads on the uncorrected exact-source control, so those fixes remain valuable independent of the 64.91 MiB/rank saving. Two intermediate candidates were rejected by independent review before final approval. The previous NaN integration blocker is resolved; no speed claim is made beyond no measurable regression.

Full chronology and evidence: https://github.com/malaiwah/glm52-exl3-vast/blob/d4b0a90/docs/field-review-results/2026-07-30-vast-46335896/UPSTREAM-REPAIR-CAMPAIGN.md#b-sparkinfer-w4a16-scratch-lifetime

Exact r14 deployable-source replay

The immutable r14 SparkInfer release tree was independently reconstructed from blackwell-llm-docker@2464cc03: base 9b852b281250123fe323f63ccb1df3cac0f3bbca plus the release lock produced the published tree 8110e3ea417794bfb08aff1fba20135102e5536b exactly. Applying the accepted #100/#102 repairs produced candidate e2205cba8d78db03427a3945a75a1f647b51ef5a (tree 1d378d599538daecfc1e8c11a44010b27d3ddfe5).

On physical GPU 2 with a fresh candidate-specific extension cache, the complete seven-file union passed 332 passed / 16 intentional skips cold in 237.03 s and identically warm in 10.03 s. This closes the image-source composition boundary for the W4 repair; no new numerical, compile-cache, capture, or mixed-K regression appeared.

Exact source provenance and immutable raw logs: r14 reconstruction ledger, cold log, warm log.

[malaiwah]

The actionable scope is implemented by SparkInfer #100 at c2f135c and caller-side vLLM #210 at 47dc47d87f428e195f66cd8e7beffd24946a415b. Measured results are 64.90904 MiB/rank of unreachable SparkInfer reserve removed and 1,120.2 MiB/rank recovered by the full-model EXL3 prefill-capacity bound. Their approximately 1.157 GiB/rank sum is expected arithmetic until a matched combined A/B is published. Evidence: https://github.com/malaiwah/glm52-exl3-vast/blob/5c76a2536e7fc9a5f1cb6bf182531889f5385e65/docs/field-review-results/2026-07-30-vast-46335896/UPSTREAM-REPAIR-CAMPAIGN.md. Keeping this issue open until both implementation PRs merge.