ops::linear q5: one Capacity=32 K-split rung serves all T<=96, so T=2 costs the same as T=17

[giveen]

The Q5 plain-linear shape dispatchers serve every T<=96 from a single
Capacity=32 K-split MMA rung:

// src/ops/linear/q5/shapes/n5120_k6144.cu
if (tokens == 1) return launch_q5_split4_c1_k6144;
if (tokens <= 96) return launch_q5_ksplit_mma<5120, 6144, 32, 96>;

Capacity is a compile-time parameter. It becomes ActiveCols in the kernel
and sizes both the accumulator fragments (kNt = kTileCols / 8) and the staged
item count (kItemsPerSplit = ActiveCols * (kTileK / 8)). Only grid.y tracks
the runtime column count. A T=2 call therefore stages and accumulates 32
columns' worth of work to use 2 of them.

The cost is flat and directly measurable. ninfer_linear_bench, N=5120/K=6144,
RTX 5090, --repeat 100, cold cache:

./build/bench/ninfer_linear_bench --qtype Q5 --n 5120 --k 6144 --sweep 1:32:1 --repeat 100

T	median_us
1	18.43
2	28.67
8	28.67
17	28.67
18	30.72

T=2 through T=17 are identical at 28.67us: the whole band pays the T=17 price.

Q4 does not have this problem — it has laddered its K-split rungs since it
landed:

// src/ops/linear/q4/shapes/n5120_k6144.cu
if (tokens <= 4)  return launch_q4_ksplit<5120, 6144, 4>;
if (tokens <= 8)  return launch_q4_ksplit<5120, 6144, 8>;
if (tokens <= 16) return launch_q4_ksplit<5120, 6144, 16>;
if (tokens <= 24) return launch_q4_ksplit<5120, 6144, 24>;
if (tokens <= 32) return launch_q4_ksplit<5120, 6144, 32>;

The Q5 linear_add route also already instantiates the same Q5 template at
8/16/24/32 (src/ops/linear_add/q5/q5_linear_add_ksplit_mma.cu). Only the Q5
plain-linear dispatchers were left on one rung, so the narrower instances are
compiled but unreachable from that path.

Laddering the rungs measures 20-40% faster across T=2..16 on all four Q5 shapes
on an RTX 5090, with T=17..32 held as an unchanged control band to bound
per-run drift.

One caveat worth stating up front: the win is real per call but does not
surface end to end on qwen3.8-27b — that decode path does not spend enough time
in these four shapes. So this is an op-level inefficiency, not a throughput bug.

Draft PR: #292, which ladders these dispatchers (and includes related Q5
small-batch linear_add routing work). Measurements, the rejected 24-rung, and
the shape-dependent T=1 boundary are documented there.

🤖 Generated with Claude Code