[perf] W4A16 launch path + indexer env parsing: full kernel rebuild before every compile-cache probe, prewarmed launches unreachable at runtime, per-call getenv on the paged-indexer prefill route

[malaiwah]

Found during a performance review of the v20 r12 release image voipmonitor/vllm:gilded-gnosis-v20-vllmb46c3aa-si35aebc6-fi801d57a-cu132-20260730-r12. Code refs are against the composed r12 tree; all cited sparkinfer files are byte-identical to public commit de04125508c75e748016493f851e6bd4513b515b, so permalinks use that SHA. vLLM caller refs are from the composed r12 vLLM tree (base b46c3aa). Deployment context: GLM-5.2 753B MoE (hidden 6144, 78 layers, top-k 8, indexer top-k 2048), EXL3 trellis 3 bpw, TP4, FULL_AND_PIECEWISE graphs; launcher defaults VLLM_EXL3_TRELLIS_MAX_M=32, VLLM_EXL3_TRELLIS_BLOCK_M=8, VLLM_EXL3_PREFILL_CHUNK=128.

All items below are per-call Python/launch-path costs: they are paid on every eager MoE/indexer invocation (prefill steps, and any batch outside graph coverage) and are amortized to capture time inside CUDA graphs. Related: #93 (trellis scratch sizing), #94 (indexer fold slab), and the vLLM-side routing-dtype/env issue local-inference-lab/vllm#204.

1. W4A16 fused-MoE runtime resolve rebuilds the full kernel object before every compile-cache probe; the plan-time prewarm is unreachable from that probe

Where (runtime resolve, no force_tile_config): sparkinfer/moe/_shared/kernels/w4a16/kernel.py#L10248-L10277

    if fused_launch is None:
        fused = compile_w4a16_fused_moe(
            size_m=m,
            ...
            # (no force_tile_config here)
        )

Where (vLLM bind always passes None): sparkinfer/moe/fused_moe/_impl.py#L937-L959 — TPMoEScratchPlan.bind(...) hard-codes fused_launch=None, topk_sum_launch=None, and the prewarmed launches stored at #L6455 (_prewarmed_fused_launches) are never read anywhere — they exist only to warm _FUSED_CACHE.

Mechanism, part A — per-call Python before the probe. compile_w4a16_fused_moe (kernel.py:7026) runs, on every run_w4a16_moe call with fused_launch=None:

• 2 (up to 4) _select_tile_config sweeps (#L7161-L7216) — these run even when the result will be overridden by a pinned tile config;
• construction of a full W4A16FusedMoeKernel, i.e. two W4A16GemmKernel inits (~300 lines of derived-field arithmetic each), including an unconditional nf3_codebook_pools(...) recompute (kernel.py:826-831) and torch.cuda.get_device_properties(torch.cuda.current_device()) per GEMM (kernel.py:894);
• only then the cache probe on kernel.__cache_key__ (L7423-7428), and on a hit a dataclasses.replace of a ~30-field frozen result (L7430-7435). The same pattern repeats for the top-k-sum launch.

Recomputed cost: a faithful pure-Python replica of the two _select_tile_config sweeps measures ~7 us per invocation on this host; the kernel-object construction around it is several times that. Estimated total ~30-100+ us per layer per eager call, x78 layers => roughly 2-8 ms of pure Python per eager prefill step, repeated every step (amortized only inside captured graphs). vLLM profiler traces of eager GLM prefill should show this directly under compile_w4a16_fused_moe.

Mechanism, part B — the prewarm can never be hit by the runtime probe. The plan-time prewarm passes force_tile_config=core_plan.trellis_tile_config (_impl.py#L6249-L6278); vLLM pins (64, 256, 64, 256) (or (64, 128, 64, 128)) via exl3.py:_trellis_tile_config. The runtime resolve omits force_tile_config, so its tiles come from _select_tile_config — and at moe_block_size=8 the candidate list is _SMALL_BATCH_TILE_CONFIGS = ((128,128,256),(64,128,128),(128,64,128)) (#L219-L223), which does not contain the pinned (tile_k=64, tile_n=256) at all (our replica selects (128,128,256) for all GLM-shaped m). Since the fused cache key embeds the FC tile shapes, cta_threads and blocks_per_sm (kernel.py:4780-4816 via the per-GEMM key L1000-L1037), the prewarmed entries and the runtime probes live under different keys: the prewarm JIT-compiles kernels that are never launched, and the first-use CuTe JIT (100s of ms per specialization) that the prewarm comment (L6452-6454) intends to avoid still happens at warmup / first live shape.

Mechanism, part C — the exact-M prewarm threshold. _impl.py#L6239-L6247:

        fused_token_counts = (
            tuple(range(1, capacity_tokens + 1))
            if capacity_tokens <= 32
            else (capacity_tokens,)
        )

At the launcher default VLLM_EXL3_TRELLIS_MAX_M=32 the decode plan prewarms every exact M (good). But the prefill plan's capacity is max_num_batched_tokens (3072/8192), so it prewarms only that single M; and any operator who raises TRELLIS_MAX_M to cover larger FULL-graph capture sizes (e.g. 64) silently flips the decode plan from "all M" to "capacity only". Because blocks_per_sm is m-dependent at small M (work-CTA clamp, kernel.py:367-369), each small M is its own cache entry — an M bucket that was neither prewarmed (part B makes that moot today) nor exercised during vLLM's warmup dummy runs pays a multi-100-ms JIT mid-serving on first occurrence.

Suggested fix (in order of impact):

1. Consume the plan's prewarmed launches: have bind() attach the matching _prewarmed_fused_launches entry instead of hard-coding fused_launch=None — this skips the entire per-call prologue and makes the prewarm meaningful.
2. Failing that: pass force_tile_config on the runtime resolve so prewarm and runtime agree on keys, and probe the cache with a key derived from the inputs before constructing kernel objects.
3. Align the prewarm threshold with the plan capacity (or log when capacity > 32 truncates coverage).

2. _get_c_tmp fallback: capture raises, eager silently allocates per call — add a log-once

Where: kernel.py#L9164-L9184

        if int(scratch.numel()) >= int(elements):
            return scratch[: int(elements)]
    if torch.cuda.is_current_stream_capturing():
        raise RuntimeError(
            "W4A16 GEMM scratch is not initialized for CUDA graph capture; ..."
        )
    return torch.empty((elements,), dtype=torch.float32, device=device)

Mechanism: if a caller-provided fc*_c_tmp binding is under-sized, capture fails loudly but eager mode silently falls into a fresh fp32 torch.empty per call per layer — allocator churn that hides the sizing bug precisely in the mode where it is cheapest to notice. Defensive today (vLLM's bindings are plan-sized), but the asymmetry means an eager-only regression would ship unnoticed. Suggested fix: logger.warning-once (with requested vs provided numel) when the fallback path is taken.

3. Paged indexer prefill route parses env per call and per supertile chunk

Where: sparkinfer/attention/nsa_indexer/paged.py#L96-L128 (fold policy, 2x os.getenv, called per index_topk_fp8 at L942->L159), #L717-L735 (supertile-K), #L1034 (per chunk), and nsa_indexer/kernel.py#L3002-L3006 (os.environ.get per run_paged_supertile_logits_kernel call at L2887).

Mechanism & scope (sharpened from our first pass): the production decode route returns at the fused early-exit (paged.py:839-865) before any env read, so decode is unaffected. The tiled/packed-contiguous route (prefill; decode only if the fused route is not resolved) pays per call: 2 getenv (fold policy) + up to 1 (supertile-K; usually short-circuited by the binding/scratch attrs at L867-870) + 2 per supertile chunk (L1034 + kernel.py:2887). Cost: 78 layers x (2 + 2 x num_chunks); at 512k context (16 chunks of 32768 tokens) that is ~2,600 os.getenv + strip/lower/int-parse per prefill step — order 1-3 ms of Python, every step, on the eager path. Fix: resolve once at plan/module init — fused_indexer.py:110 already does exactly this for its own flag, and the vLLM side has the same pattern (tracked in local-inference-lab/vllm#204).

Note on routing-id dtype (cross-ref vllm#204)

For completeness: the int64->int32 topk cast quoted in earlier drafts of this review (kernel.py:10115-10117) is gated on weight_layout in ("packed", "nf3_2p1") and m<=8, so it is unreachable on the GLM trellis3_t256 path; the route-pack/top-k-sum kernels consume int64 ids natively via route_ids_dtype-specialized launches (prewarmed for both dtypes, _impl.py:6597). The sparkinfer-side residual cost of vLLM handing int64 is therefore 2x route-id bytes plus duplicate kernel specializations, not a per-step cast. The root fix is on the vLLM side (local-inference-lab/vllm#204).

Verification hint

nsys/py-spy an eager GLM prefill step: expect visible per-layer time under compile_w4a16_fused_moe / W4A16GemmKernel.__init__ (item 1) and _read_two_level_fold_policy / os.getenv (item 3). For item 1 part B: log _FUSED_CACHE keys after plan-time prewarm vs after the first live forward — the prewarmed keys are never probed.

[malaiwah]

Independent re-review by a second reviewer (did not author the original claims), against the same trees (de04125 sparkinfer / composed r12 vLLM), re-deriving every number from the shipped configs and running the arithmetic inside the r12 image itself — retractions included.

All measurements below were taken by importing the r12 image's own sparkinfer on a 4×RTX PRO 6000 Blackwell host (sms=188, shared_memory_per_block_optin=101376), with the real model geometry read from the checkpoint config: hidden_size=6144, moe_intermediate_size=2048 → I_tp=512 at TP4, n_routed_experts=256, num_experts_per_tok=8, first_k_dense_replace=3 → 75 MoE layers (not 78), indexer_types = 21 full / 57 shared, index_topk_freq=4 → 21 indexer-computing layers (not 78).

Verdicts

Claim	Verdict	Note
1A: compile_w4a16_fused_moe builds the full W4A16FusedMoeKernel (2× W4A16GemmKernel) before the _FUSED_CACHE probe	HOLDS	Verified: sweeps L7161-7216, force-pin L7351, kernel ctor L7391, probe L7423-7428. Unconditional nf3_codebook_pools (L826-831) and get_device_properties (L894) per GEMM also confirmed uncached
1A: "2 (up to 4) sweeps"	HOLDS-WITH-CORRECTION	Exactly 2, never 4, on this config: FC1/FC2 both select cta_threads=256 for every m on both plans, so the L7187 re-sweep never fires
1A: sweeps ≈ 7 µs	HOLDS-WITH-CORRECTION	Measured 14.3 µs for the two real _select_tile_config calls (2×)
1A: 30–100+ µs per layer, 2–8 ms per eager prefill step	HOLDS-WITH-CORRECTION	Measured 54.2 µs/call cache-hit, ×75 layers = 4.07 ms per eager prefill step. Inside the filed range; layer count 78→75
1A: eager-only, amortized inside captured graphs	HOLDS	Turnkey decode m=48 is inside CUDAGRAPH_CAPTURE_SIZES=4..64; prefill (m up to 3072) is uncaptured → pays per step
1B: bind() hard-codes fused_launch=None	HOLDS	_impl.py:957-958, and api.bind is a pass-through to plan.bind
1B: _prewarmed_fused_launches "never read anywhere"	HOLDS (literally) / reframed	Only assigned (_impl.py:6455) and never read — but that is deliberate: it is a keepalive, the compile cache is the intended channel (see quote below). Presenting it as evidence of deadness misreads the design
1B: prewarm keys "can never match" because the pinned tile is absent from the candidate list	HOLDS for the decode plan / RETRACT for the prefill plan	At block_m=64 the runtime sweep independently returns exactly the pin (64,256,64,256). The tile argument is only valid at block_m=8
1B: net effect — prewarmed launches are unreachable from the runtime probe in the shipped configs	HOLDS, different mechanism	The general blocker is max_m_blocks, not the tiles: it enters the GEMM cache key and is derived from the live pow2 route-pack bucket. Keys match only when live m == plan capacity
1C: MAX_M=64 flips the decode plan from all-M to capacity-only	HOLDS, and it is the shipped state	Turnkey sets VLLM_EXL3_TRELLIS_MAX_M=64 → capacity_tokens=64 > 32 → fused_token_counts=(64,). Not hypothetical
1C: "each small M is its own cache entry because blocks_per_sm is m-dependent"	RETRACT (reason wrong)	blocks_per_sm == 1 for every m on both plans (uses_m_block_8→1 at block_m=8; register+smem limit→1 at block_m=64). Per-m key divergence is entirely max_m_blocks
Fix 1 (bind attaches the prewarmed launch)	viable	Capture-safe, passes the existing contract check, numerically neutral on this path — see fix-safety
Fix 2 as filed ("pass force_tile_config on the runtime resolve so keys agree")	RETRACT	Measured 6% slower per call (57.7 vs 54.2 µs) and still misses on max_m_blocks. Second half of that bullet ("probe with a key derived from the inputs") is the real fix
Fix 2b: probe key computable without the sweep	RETRACT as stated / correct form given	The __cache_key__ contains the sweep outputs (tile_n/tile_k/cta_threads/shared_words/blocks_per_sm), so it is provably not hoistable. An args-level memo is, and is exactly equivalent
2 (GG-34): _get_c_tmp raises under capture, silently torch.emptys per call eagerly	HOLDS	Code quote exact (L9170-9184). Confirmed unreachable in both shipped configs (arena fc*_c_tmp is sized from the plan's route_slots_capacity ≥ every live pow2 bucket), i.e. defensive-only, as filed
3 (GG-38): reads on the tiled/packed route	HOLDS-WITH-CORRECTION	1 read per supertile chunk, not 2 — kernel.py:3002 is the only per-chunk read; there is no kernel.py:2887 read
3 (GG-38): "production decode returns at the fused early-exit before any env read"	HOLDS at the gate / RETRACT for turnkey	_GLM32_FUSED_MAX_ROWS=16; turnkey decode is 8×(1+5)=48 rows → resolve_fused_indexer_path(...) returns False → route is paged_tiled, which does read env. Mitigated only because m=48 is a captured graph size
3 (GG-38): ~2,600 getenv/step, 1–3 ms	HOLDS-WITH-CORRECTION	126–378 reads/step and 0.05–0.14 ms/step — 7–12× fewer reads, ~10–20× less time
Note on routing-id dtype (GG-33 retraction)	HOLDS	Re-verified: use_tc_decode/direct_topk_eligible both gate on weight_layout in ("packed","nf3_2p1") (L10107, L10122), so the L10115-10117 cast is unreachable for trellis3_t256

1B — key-construction evidence (this is the load-bearing correction)

Two independent components decide the match. I constructed both keys with the real W4A16FusedMoeKernel in the r12 image.

(i) Tiles. The runtime sweep is not blind to the pin — at block_m=64 it reproduces it:

block_m= 8  m=1..64    -> (fc1_tk,fc1_tn,fc2_tk,fc2_tn) = (128,128,128,128)   match_pin=False
block_m=64  m=65..8192 -> (fc1_tk,fc1_tn,fc2_tk,fc2_tn) = (64,256,64,256)     match_pin=True

The decode plan uses block_m = VLLM_EXL3_TRELLIS_BLOCK_M = 8 → cta_m_blocks=1 → _SMALL_BATCH_TILE_CONFIGS, whose first fitting entry is (tile_k=128, tile_n=128, 256). Because uses_m_block_8 pins blocks_per_sm_limit = 1 (kernel.py:351-361), every candidate ties at 1 and the strict > at L480 keeps the first — the sweep has no opinion at all here, the result is list order. The prefill plan uses VLLM_EXL3_PREFILL_BLOCK_M=64 → cta_m_blocks=4 → _LARGE_BATCH_TILE_CONFIGS, whose first entry is (64, 256, 256) = the pin (hidden 6144 % 256 == 0, I_tp 512 % 256 == 0 → exl3.py:1393-1394 returns (64,256,64,256)).

(ii) max_m_blocks — the real blocker. required_m_blocks = block_expert_ids.numel() (kernel.py:10201), and route_pack.pack_topk_routes_by_expert slices that buffer to a capacity derived from route_pack_token_capacity = next power of two of the live token count (host.py), falling back to the exact live numel only when the arena is too small. So max_m_blocks — which is in W4A16GemmKernel.__cache_key__ (kernel.py:1030) — tracks the live m bucket, while the prewarm always compiles at plan capacity:

[prefill turnkey] cap=3072 block_m=64 arena_m_blocks=636
   m=  65  live=268  tiles=(64,256,64,256)  KEY_MATCH=False  (max_m_blocks 268!=636)
   m= 128  live=268  tiles=(64,256,64,256)  KEY_MATCH=False  (max_m_blocks 268!=636)
   m= 512  live=316  tiles=(64,256,64,256)  KEY_MATCH=False  (max_m_blocks 316!=636)
   m=1024  live=380  tiles=(64,256,64,256)  KEY_MATCH=False  (max_m_blocks 380!=636)
   m=2048  live=508  tiles=(64,256,64,256)  KEY_MATCH=False  (max_m_blocks 508!=636)
   m=3072  live=636  tiles=(64,256,64,256)  KEY_MATCH=True
[decode turnkey]  cap=64  block_m=8  arena_m_blocks=288
   m=1..64: tiles=(128,128,128,128) != pin AND live max_m_blocks in {8,16,32,64,128,256,288} -> KEY_MATCH=False everywhere

Same shape at the gate (cap=8192: match only at m=8192; cap=32 decode: no match at any m).

So the corrected statement of 1B is: the decode plan's prewarm is 100% dead (two independent key mismatches), and the prefill plan's single prewarmed entry is live only for full-capacity chunks. The prewarm is not a vestigial artifact — the intent is explicit and the mechanism is sound, it just does not survive the two divergences:

# _impl.py:6451-6454
# Keep strong references to the launches primed in the module caches,
# but leave the runtime binding unresolved.  ``run_w4a16_moe`` uses a
# None launch as a dispatch signal and then gets these exact objects
# from its compile cache without doing first-use JIT work.

Consequence flip worth adding to the issue. Because the mismatch is bucket-shaped rather than absolute, the cost is not only the Python prologue:

• Startup: the decode plan compiles up to 32 CuTe kernels (fused_token_counts = range(1, 33) at MAX_M=32) that can never be probed — pure wasted startup compile.
• Mid-serve: every unseen pow2 prefill bucket (128/256/512/1024/2048/4096) is a fresh _FUSED_CACHE key on first occurrence. vLLM's dummy run warms only the capacity bucket, so a 700-token or 1500-token prompt is a first-use compile after the engine is up (disk-cache-backed by SPARKINFER_COMPILE_CACHE_DIR after the first time on a host). This is checkable directly from existing r12 logs: sparkinfer already logs post-engine-start compiles by default (compiler.py:689-697, gated on the vLLM engine-started marker; SPARKINFER_LOG_CUTE_COMPILES_AFTER_ENGINE_START).
• Latent capture hazard: compile_w4a16_fused_moe has no is_current_stream_capturing() guard (unlike the planner at _impl.py:6185). Any capture size whose bucket was not warmed by a preceding eager run would attempt a CuTe compile inside capture.

1A — cost, re-measured

Per call, with a warm cache (dummy result injected so the probe hits and no compile runs), at the production geometry:

Component	µs/call	×75 layers
2× _select_tile_config	14.3	1.07 ms
W4A16FusedMoeKernel(...) (2× GEMM ctor)	24.5	1.84 ms
validation/normalisation + probe + dataclasses.replace	~15.4	~1.16 ms
compile_w4a16_fused_moe total (cache HIT)	54.2	4.07 ms
of which pure recompute: nf3_codebook_pools×2 + get_device_properties×2	6.7	0.50 ms

m-invariant (39 µs pre-probe at m=128 and at m=3072 alike), so this is a flat per-step tax whose relative weight is worst on small prefill chunks. Turnkey: 75 layers × 1 call/step (the EXL3 prefill path issues one bind+run for the whole chunk — VLLM_EXL3_PREFILL_CHUNK only bounds the parity path), +1 for the MTP draft layer. Gate: same per-step figure. Decode: paid once per captured size at capture, then zero.

GG-38 — recount and scope

Per index_topk_fp8 call on the non-fused route: 2 (_read_two_level_fold_policy, paged.py:97+112) + 0–1 (_resolve_supertile_k, paged.py:719 — normally short-circuited by binding.supertile_k/scratch.paged_tile_logits_k_rows) + 1 per supertile chunk (_env_indexer_stream_scorer_enabled, kernel.py:3003, called at paged.py:1034). Measured in-image: 1.030 µs for the fold-policy pair, 0.350 µs for the per-chunk read.

num_chunks = ceil(real_page_table.shape[1] / (32768/64)) — driven by block-table width, not live seqlen:

Config	chunks	layers	reads/step	Python/step
131072 ctx	4	21	126	0.05 ms
524288 ctx	16	21	378	0.14 ms
(as filed: 78 layers, 2/chunk, 512k)	16	78	~1,404–2,600	0.5 ms

Scope correction: resolve_fused_indexer_path(topk=2048, num_heads=32, ...) gates on _GLM32_FUSED_MAX_ROWS = 16, and the indexer plan is built with max_q_rows = q_fp8.shape[0] (sparse_attn_indexer.py:1093). Turnkey decode is max_num_seqs 8 × (1+MTP5) = 48 rows → returns False → route = paged_tiled, so the "decode returns before any env read" sharpening does not hold at turnkey (it does at the gate: seq=1 × (1+MTP3) = 4 rows → fused). It is harmless in practice only because m=48 is inside CUDAGRAPH_CAPTURE_SIZES=4..64, i.e. the reads are amortized to capture, not skipped. That distinction matters for any future uncaptured decode step.

Fix-safety notes

Fix 2b, corrected form — args-level memo in front of compile_w4a16_fused_moe (recommended). The probe key is provably not hoistable above the sweep: __cache_key__ embeds tile_n, tile_k, cta_threads, shared_words, blocks_per_sm — all sweep outputs. But the function is a pure function of its keyword arguments plus torch.cuda.current_device() (already the device element of the existing key); sms/max_shared_mem are explicit args. A dict keyed on that argument tuple, storing the post-replace result, is exactly equivalent:

• Cannot change kernel selection — a hit returns precisely what the same arguments produced on the first call.
• Capture-safe — dict lookup, no allocation; strictly safer than today (removes the unguarded compile-inside-capture path above).
• Semantics-identical on repeats — the _small_m_direct_supported prewarm side effect at L7437-7469 is already skipped on a _FUSED_CACHE hit (return replace(...) precedes it), so a front memo changes nothing observable. Validation raises are deterministic in the same args.
• Numerics — untouched; same compiled object, same tiles.
• Also removes the per-call 30-field dataclasses.replace allocation.
• Expected win: 54.2 → ~1 µs/call, i.e. ~4 ms/eager prefill step recovered. Measure: A/B a fixed-length prefill sweep (256/1024/3072 tokens) on prefill TTFT, and confirm with py-spy that compile_w4a16_fused_moe / W4A16GemmKernel.__init__ vanish from the eager step.
• An lru_cache on _select_tile_config alone recovers only 14.3/54.2 = 26% — don't stop there.

Fix 1 (bind attaches the prewarmed launch). Works, with caveats:

• Passes the existing contract validation (L10284-10347): the compared tuple excludes tiles, and max_m_blocks is only checked with < against required_m_blocks, which the capacity prewarm satisfies.
• Capture-safe — an m-keyed dict lookup in bind() keeps the documented allocation-free/views-only contract.
• Numerically neutral on this path, which is the non-obvious part: trellis3_t256 forces schedule_whole_tiles=True (kernel.py:4669-4671), and that path takes tail_mn_tiles = 0 — "one CTA computes each mn-tile over the full K … No split-K tail, no lock traffic" (kernel.py:1300-1305). With no split-K there is no cross-CTA partial reduction whose tree could change, so moving decode from tile_k=128 to tile_k=64 alters only staging granularity, not accumulation order. [INFERENCE] on bitwise equality — worth one oracle run before trusting it, but there is no split-K mechanism for it to break.
• It does change which kernel decode runs ((128,128) → (64,256)). Since all block_m=8 candidates tie at blocks_per_sm=1, today's choice carries no measured justification either way — so this is a genuine A/B, not a regression risk: gate MTP0/MTP3 decode tok/s (48.52 / 100.82) plus acceptance (0.64865, re-baseline if the oracle check is skipped).
• Grid work is not de-specialized by attaching a capacity launch on prefill: route_blocks comes from the device-side packed_route_count (kernel.py:1291), not from max_m_blocks.
• On its own this fix is worth less than the memo: it removes the prologue and the bucket compiles, but requires the prewarm to be made bucket-aware (below) to actually cover live shapes.

Fix 3 / prewarm alignment. Two changes needed for the prewarm to be reachable at all: prewarm the pow2 max_m_blocks buckets (not just capacity), and either drop force_tile_config from the prewarm or apply it in the runtime resolve. Dropping the pin is the cheaper direction — it is a no-op for the prefill plan (the sweep already returns the pin) and makes decode agree by construction. Note force_tile_config is validated (_candidate_tile_fits, L7372-7390) but is otherwise unconstrained by the weight packing on this layout: the prepared-vs-compiled tile assertion at L10349-10361 is nf3_2p1-only, so there is no packing invariant tying trellis3_t256 weights to a tile width.

GG-34. logger.warning-once on the fallback branch is safe: the branch already raises under capture, so the added call can never execute during capture; a module-level bool costs nothing on the hot path.

GG-38. Resolving both env values once at plan/module init (as fused_indexer.py:110 already does) has no ordering or capture implications — the values are capture-time-invariant by construction. One caveat: _env_indexer_stream_scorer_enabled is currently re-read inside the chunk loop, so hoisting it changes the error-raising point at L1040-1044 from per-chunk to per-call; that's a strict improvement but worth keeping the message's chunk_width_tokens context.

Priority triage

Trivial + safe (ship without measurement):

• Args-level memo in front of compile_w4a16_fused_moe (~4 ms/eager prefill step, no numerics/capture/ordering exposure).
• Hoist the two indexer env reads to plan/module init (~0.05–0.14 ms/prefill step; kills a per-chunk read in a loop).
• log-once in _get_c_tmp.
• Add an is_current_stream_capturing() guard to compile_w4a16_fused_moe (or make the miss path raise) — closes the latent compile-inside-capture hazard for free.

Needs measurement:

• Making the prewarm reachable (bucket-aware max_m_blocks + drop the pin, or Fix 1). Real win is the mid-serve first-use compiles; size it first from the existing post-engine-start compile log before changing the decode tile config.
• Fix 1's decode tile change: A/B gate MTP0/MTP3 tok/s + acceptance.

Drop:

• Fix 2 as filed (pass force_tile_config on the runtime resolve) — measured 6% slower and still misses.
• The "each small M is its own cache entry because blocks_per_sm is m-dependent" rationale — blocks_per_sm is 1 everywhere here.
• The tile-mismatch framing for the prefill plan — the sweep already returns the pin there.