Muse Glimmer's vision tower runs 50 layers on the naive attention kernel: ~4.8 s/image at 1024 tokens, unreached today and a 5 s TTFT stall the moment W3 lands

[localai-bot]

src/vllm/model_executor/models/muse_glimmer_vision.cpp:639 calls
vt::Attention, the "correctness-grade" naive kernel
(src/vt/cuda/cuda_ops.cu:1463, self-described at :1456-1459). It is an
instance of the class issue #1544,
and it is the worst-shaped one in the tree.

Why this one is bad

Verified in source: it is the sole attention path in the tower, non-causal,
behind no knob or A/B rung, across 50 layers, H=16, head_dim=96.

Cost model Q * H * K * 5.70 ns (validated within 7% against two recorded
measurements — see #1544):

tokens	per call	per tower forward (50 layers)
1024	95.6 ms	4.8 s
4096	~1.5 s	~34 s

For scale, the Qwen3-VL tower fix was justified by 1.51 s and delivered
14.3x. This is 3.2x worse at 1024 tokens, in a tower with nearly twice
the layers.

It is not reachable today, which is exactly why to fix it now

MuseGlimmerEncodePixelGroups and MuseGlimmerGenerateGreedyViaRegistry have no
caller in src/, examples/ or include/vllm.h — only tests/. The registry
says so directly at muse_glimmer_registry.cpp:13-14: "The perception encoder is
still W3, so an image or video prompt is a pending brick."

So this is a landed-unreached slice (AGENTS.md "Nothing lands dead") that becomes
a ~5 second TTFT stall the moment W3 is wired, discovered by whoever wires it
rather than by whoever caused it. Fixing it before that wiring costs one line;
fixing it after costs a regression hunt.

The fix

Route to vt::AttentionDenseFlash. head_dim 96 needs 2*64*96*2 = 24 KB of
dynamic shared memory, inside the 48 KB default cap (see #1544 item 2 — the op's
advertised <= 256 ceiling is wrong, but 96 is safely under the real one).
AttentionDenseFlash honours causal (cuda_ops.cu:3277-3296) and Muse's
block-diagonal mask is already expressed as per-segment row slices that map 1:1
onto it, so masking is not a blocker. AttentionDenseFa2 is NOT usable —
head_dim != 64 (cuda_flash_attn_fa2.cu:557-560).

Also owes a ## Owed entry on the Muse Glimmer spec, since the tower is an
unreached staged slice.

Labelled uncertainty

Verified in source: sole path, non-causal, no knob, H=16, head_dim=96,
no production caller.

Inferred, NOT measured: the token count — there is no image processor or
resizer for Muse Glimmer in the tree; the tower takes
MuseGlimmerVisionImage{height,width} directly and only tests construct one
(12x12 and 8x8 synthetic patches, test_muse_glimmer_vision.cpp:127-132). The
1024/4096 anchors correspond to 448x448 and 896x896 at patch_size=14 and are
illustrative, not the shipped resolution. The 13-full/37-window layer split is
also inferred from the fallback rule at muse_glimmer_weights.cpp:429-439; the
comment states the released config.json ships an explicit layer_types, which
was not read. The 5.70 ns constant transferring to head_dim 96 is inferred from
anchors at 64 and 72.

Whoever takes this should pin the real resolution first — it moves the estimate
quadratically.

Found while auditing the class after LTX-2.5's DiT forward was traced to this
kernel.

[localai-bot]

The resolution is pinned, and the estimate moves up rather than down. This issue
asked whoever took it to pin the real resolution first; it was cheap, because the
released checkpoint is already on the NAS at
/mnt/nas_share/checkpoints/muse-glimmer-30b/ and needed no download.

Two of the three labelled uncertainties are now resolved

The 13-full / 37-window split is READ, not inferred. config.json
vision_config.layer_types ships explicitly, length 50, with full_attention
at indices 3, 7, 11, ..., 47, 49. That is exactly what
muse_glimmer_weights.cpp:429-439 computes when the key is absent, so the
fallback rule is confirmed correct against the shipped file rather than merely
plausible.

The window is 32x32 = 1024 patches. pos_emb_height and pos_emb_width are
both 32, and MuseGlimmerVisionSparsePermutation
(muse_glimmer_vision.cpp:355-382) segments on them. This issue's cost model did
not carry it: a window layer does not attend over the whole image.

The token count is still two readings, and BOTH are worse than 4.8 s

processor_config.json sets image_processor.max_image_tokens = 4096, and the
checkpoint README's overview table says "Max visual tokens per image | 4,096".
Whether that counts post-merge tokens (what the language model sees) or patch
tokens (what the tower runs on) is not decidable from the checkpoint alone:
there is no image processor for this model in the tree and no local
transformers checkout.

reading	patch tokens	grid	windows	naive cost per tower forward
A: 4096 post-merge (merge 2, so x4)	16,384	128x128	16	375 s
B: 4096 patch tokens	4,096	64x64	4	34 s
this issue's illustrative 448x448	1,024	32x32	1	4.8 s

At Q * H * K_eff and the 5.70 ns/iteration GB10 constant:

reading A: 13 x (16384 x 16 x 16384) + 37 x (16 x 1024 x 16 x 1024)
         = 5.583e10 + 9.932e9 = 6.576e10 x 5.70 ns = 375 s

reading B: 13 x (4096 x 16 x 4096) + 37 x (4 x 1024 x 16 x 1024)
         = 3.489e9 + 2.483e9 = 5.973e9 x 5.70 ns = 34.0 s

Reading A is the more likely one, and the architecture is the argument.
window_attention over a 32x32 block does nothing at all unless the grid exceeds
32x32. At 448x448 the grid is one window, so all 50 layers compute the identical
thing and 37 of the checkpoint's layer-type entries are inert. A tower that ships
37 window layers is meant to run at grids well past one window.

The 5.70 ns constant is still anchored at head_dim 64 and 72 and its transfer to
96 is still inferred, as #1544 labels it. Nothing here depends on it being right
to better than a factor.

Fixed in #1579

muse_glimmer_vision.cpp:639 now names vt::AttentionDenseFlash. It lands
UNREACHED under the .agents/reachability.md staged-slice exception, and the
missing wiring is now tracked by #1566 because the model's umbrella #268 was
closed and nothing open named the gap.

Two things that change nothing about the fix but are worth recording. On CPU the
swap is byte-identical by construction, because cpu_ops.cpp:3760 registers
kAttentionDenseFlash to the same AttentionKernel pointer kAttention gets at
:3750; every golden holds to the digit. On CUDA it is not bit-identical, and
no CUDA run backs the change, because dgx:gpu0 was held and no lease was taken.
The A/B is recorded as owed in the row's spec rather than claimed.