[KV Connector] Add canonical KV layout fields for TP-agnostic offload

[Etelis]

Extends CanonicalKVCacheRef with the per-layer intra-block layout needed to offload KV in a tensor-parallel-agnostic format, so a cache offloaded under one TP size can later be loaded under another by slicing on the global head dimension.

A head-sliceable block is (block_size, n_heads, head_bytes):

• n_heads — global KV head count; 0 = no head decomposition
• h_stride — bytes per head per token (the head-slice unit)
• bs_stride — bytes per token (n_heads * h_stride)
• replicated — meaningful when n_heads == 0: True = page is identical on every TP rank (MLA latent; any one rank's copy is complete), False = rank-specific shards (Mamba states, packed blocks; no single rank's page is complete)

register_kv_caches populates these per layer from each KVCacheSpec (h_stride = real_page_size_bytes / (block_size * num_kv_heads), which is TP-invariant). The fields live on the per-layer ref rather than the tensor, since layers of different types (e.g. attention + mamba in hybrid models) can share one tensor.

Population fails closed to the opaque default (n_heads=0, replicated=False) whenever head slicing would not be valid: DCP token-sharding, KV head replication (tp > total heads) or per-layer head count mismatch, per-token-head quant scales stored outside the page, or a page that does not divide evenly into head cells.

Schema-only: all fields have inert defaults and nothing reads them yet.

Backend KV cache layouts

Source layouts the canonical fields normalize. Per-head backends differ only by
dim order / K-V packing and reduce to n_heads = total; MLA latent and SSM
state carry no head dim and stay head-less.

Family	Backend(s)	get_kv_cache_shape
Per-head	FlashAttention	(nb, 2, bs, H, D)
	FlashInfer	(nb, 2, bs, H, D) [1]
	Triton	(nb, 2, bs, H, D+scale)
	ROCm	(2, nb, bs, H, D) [2]
	ROCm AITER FA / Unified	(nb, 2, bs, H, D)
	FlexAttention	(nb, 2, bs, H, D) [3]
	TurboQuant	(nb, bs, H, slot) [4]
	CPU	(nb, H, bs, 2·D) [5]
DiffKV	FlashAttn / Triton DiffKV	(nb, bs, H, D+Dv)
MLA	dense (triton, flashmla, cutlass, flashinfer, aiter, rocm, flashattn, tokenspeed)	(nb, bs, D)
	sparse (flashmla, FI-sm120, SWA)	(nb, bs, 656|584) [6]
	sparse (FI, rocm-aiter, xpu), indexer	(nb, bs, D)
SSM	Mamba2	conv ((I+2GN)/tp, K−1) + state (nh/tp, hd, N)
	Mamba1 / ShortConv	conv (I/tp, K−1) (+ ssm (I/tp, N))
	GatedDeltaNet	conv ((2·hk·nk+hv·nv)/tp, K−1) + state (nv/tp, hv, hk)
	LinearAttn	state (nh/tp, hd, hd)

Legend: nb=num_blocks, bs=block_size, H=num_kv_heads (per-worker), D=head_size,
Dv=head_size_v; SSM: I=intermediate, nh=num_heads, hd=head_dim, N=state_size,
K=conv_kernel, G=n_groups, tp=TP size.

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @Etelis.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[Etelis]

Closed as #48408 implements that.