GFX1100-TG200 campaign: Qwen3.5-4B Q4_K_M at >= 200 tok/s single-stream decode on the RX 7900 XTX

[ghazni101]

Goal

Serve /home/ghazni/models/vllm.cpp/Qwen3.5-4B-Q4_K_M.gguf at >= 200 tok/s text-generation throughput on the RX 7900 XTX (gfx1100), pure autoregressive greedy decode, no MTP/speculative path in any measurement arm.

Developer-set goal, 2026-08-22. Feasibility is settled and is not relitigated inside the campaign:

• llama.cpp already sustains ~200 tok/s on this exact checkpoint and this exact GPU with a q8 KV cache; 200 tok/s is demonstrated, not theoretical.
• This engine's own lm_head kernel already streams weights at ~598 GB/s on this board (TG150 evidence): the memory system delivers.
• Ceiling arithmetic: ~960 GB/s peak DRAM / ~2.2 GB weights+KV read per token ~= 430 tok/s theoretical, so 200 tok/s sits at ~47% of peak. The campaign's job is to close the gap between current position and a number physics already permits.

Starting position (measured, TG150 campaign)

Branch row/ROCM-QUANT-GEMM-BW head 094f60362 (5 commits, pushed to origin): branch-free Q6K scalar decode, split-K GEMM arm, f32-query DecodeGQA attention, row-permuted keep-quant in_proj, tiny-N f32-out GEMV.

Remaining measured per-token budget from the TG150 captures:

Item	ms/token
GdnPostConv (grid=1-block pathology)	~4.1
dispatch gap (host-bound, HIP-graph territory)	~3.0
GdnScan	~1.1
residual quant-GEMM arms below 300 GB/s effective	remaining

Current head: ~27.6 tok/s wall.

Working rules

1. Never push or merge to main on either remote; all work lands on row/* branches pushed to ghazni101/vllm.cpp only (developer directive 2026-08-22).
2. Every GPU command goes through /home/ghazni/gpu-coord/gpu-ctl run/reserve/status. Another agent shares this GPU.
3. Correctness gates are non-negotiable: op-level NMSE vs CPU oracle, token coherence on every A/B, near-tie adjudication recorded when reduction order changes.
4. Every change is A/B'd on the acceptance workload before it counts; medians, not best-case runs.
5. Attribute before optimizing: one rocprofv3 capture per head, per-kernel budget table, attack the top item. No speculative rewrites.

Acceptance gate

Median of >= 5 repetitions, idle host, gpu-ctl lock held for the whole window, batch 1, ~512-token real prompt, 256 generated tokens, greedy (--temperature 0 --seed 0), through the production entry point (examples/vllm-cli). Recorded axes: output tok/s (the gated number), steady-state TPOT, peak VRAM. Token identity vs pre-campaign output asserted for bit-exactness-claiming levers.

Definition of done

The acceptance gate passes at >= 200 tok/s median with coherent greedy output; evidence recorded per .agents/verification.md; docs updated; work merged to a task branch ready for review.

Owning matrix row: BACKEND-ROCM. Spec: .agents/specs/gfx1100-tg200.md.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:ox-alpha [omp]