Restore sparse-MLA indexer decode on gfx1151

[randomvariable]

Context

gfx1151 DeepSeek-V4 sparse-MLA decode was returning wrong top-k selections, silently, because the indexer cache is written in a shuffled 16x16 packed layout that neither the AITER stage-1 route nor the Torch reference understands. Those routes now raise rather than misselect, so the path is closed rather than incorrect. See #23 for the diagnosis.

This issue tracks reopening it. It only matters if we intend to serve a DeepSeek-V4-class model on Strix Halo — worth deciding that before spending on it.

Options

1. Shuffle-aware reads in the stage-1 and Torch routes — teach them the [page_token_tile, dim_tile, token_lane, dim_lane] layout and the trailing FP32 scale region.
2. A dedicated gfx1151 kernel, per the scoping already done: cast FP8 to BF16 in registers, WMMA-compatible 16-multiple tiles throughout, and apply the FP32 K scale after head reduction rather than dequantising the cache up front. Estimated 1-3 engineer weeks.
3. Forcing an unpacked indexer cache, which costs page-table and allocator efficiency and does not address FP8 compute.

Acceptance must include exact top-k match rate and top-k intersection against an independently written unpacking oracle — not against the current Torch path, which is itself wrong for this layout. Elementwise closeness is not sufficient, because the failure mode is ranking drift.

Worth deciding whether we care: this only matters if we intend to serve a DeepSeek-V4-class model on Strix Halo.

Acceptance

Because the failure mode is ranking drift rather than magnitude error, elementwise closeness is not sufficient evidence:

• Exact top-k match rate and top-k intersection against an independently written unpacking oracle. Not against the current Torch path, which is itself wrong for this layout — that comparison would be circular.
• Selected-score regret: the reference score lost to any changed selection.
• Coverage of partially-filled final pages, non-contiguous block tables, and token counts crossing a 16-token tile boundary.
• End-to-end output comparison at long context, not only kernel tests.

Validation / Evidence

• Focused gfx1151 run: .venv/bin/python -m pytest -q -rs tests/kernels/attention/test_rocm_indexer_cache_layout.py::test_shuffle_layout_roundtrip tests/kernels/attention/test_rocm_indexer_cache_layout.py::test_shuffle_layout_is_not_row_major tests/kernels/attention/test_rocm_indexer_cache_layout.py::test_writer_leaves_unmapped_pages_untouched tests/kernels/attention/test_rocm_indexer_cache_layout.py::test_negative_slots_are_skipped tests/kernels/attention/test_rocm_triton_attn_dsv4.py::test_compute_global_topk_ragged_indices_and_indptr tests/kernels/attention/test_rocm_triton_attn_dsv4.py::test_sparse_attn_decode_ragged_kernel tests/kernels/attention/test_rocm_triton_attn_dsv4.py::test_paged_mqa_logits_torch_handles_shuffled_layout_off_fused_path tests/kernels/attention/test_rocm_triton_attn_dsv4.py::test_paged_mqa_logits_torch_reads_multi_token_shuffle_layout tests/kernels/attention/test_rocm_triton_attn_dsv4.py::test_deepseek_v4_indexer_cache_block_size_is_64 -> 31 passed, 0 skipped, 15 warnings in 16.88s.
• Proven: shuffled 16x16 layout at block sizes 16/64/256; partial final pages 64/83; partial 16-token tile 64/20; 16-token boundary 16/40; non-contiguous descending physical page mappings; untouched-page protection; exact ragged global top-k slot/index/indptr construction against independent oracle; sparse decode from FP8 packed main/extra caches against independent reference; gfx1151 non-MI300 dispatch selects Torch shuffled-cache fallback; reachable DeepSeek-V4 indexer storage block size 64.
• Remaining blockers: no model-level DeepSeek-V4 long-context inference/model artifact available locally; no end-to-end output comparison; no published exact top-k match/intersection, selected-score regret, or final-page/non-contiguous/16-token coverage from real model execution. AITER metadata validation unavailable because supported AITER installation is absent; split-K skips are expected for gfx1151 and irrelevant to fallback path.

Notes

• Only decode is affected. Prefill's cp_gather_indexer_k_quant_cache_triton takes a LAYOUT argument and de-shuffles into a contiguous buffer first, so it was correct throughout.
• The fused gfx942/gfx950 path is told the packing via Preshuffle/KVBlockSize and handles it correctly, so it is untouched.
• Relevant to Attention parity: run every attention type on the best available path on both targets #23 (attention parity), since a target that cannot serve an attention type at all is the widest possible version of that gap.

[randomvariable]

Status: partial implementation shipped.

dfff80ad6 restores the shuffled 16x16 FP8 indexer-cache Torch path. tests/kernels/attention/test_rocm_triton_attn_dsv4.py now uses an independent packing/read oracle and covers multi-token speculative rows, ragged contexts, non-contiguous block tables, tile boundaries, scores, and exact top-k selection. Focused local regressions pass (82 passed, 21 skipped). #26 remains open because acceptance still requires hardware end-to-end long-context output comparison on gfx1151, including partially filled final pages and selected-score regret against the independent oracle. Next gate: publish that run.

[randomvariable]

Validation update:

• .venv/bin/python -m pytest tests/kernels/attention/test_rocm_triton_attn_dsv4.py -v — 66 passed, 21 skipped.
• .venv/bin/python -m pytest tests/kernels/attention/test_rocm_triton_attn_dsv4.py::test_sparse_attn_decode_ragged_kernel -v (gfx1151 GPU) — passed.
• AITER metadata-path validation unavailable in this environment.
• Model-level/end-to-end DeepSeek long-context comparison was not run, so acceptance remains incomplete: no published exact top-k match/intersection, selected-score regret, and end-to-end evidence for partial final pages, non-contiguous blocks, and 16-token boundary coverage.

Issue remains open.