qwen36: Vulkan expert tier, and staged device-local uploads for cards without Resizable BAR

[crichalchemist]

Summary

Two things, one PR because the second is what makes the first measurable on most older discrete cards.

Qwen3.6 Vulkan expert tier. make qwen36 VK=1 builds the existing qwen36_tier.c against the shared Vulkan backend. The placement logic (warmstart, heat file, upload queue, hit/miss accounting) is unchanged; a compile-time shim of twelve one-line wrappers selects CUDA or Vulkan (#if COLI_CUDA / #elif COLI_VULKAN, CUDA wins if both). On Vulkan the tier is single-device and fills once at warmstart: the Vulkan weight arena never reclaims a freed slice, so runtime LFRU swaps would leak one expert of VRAM each. Heat still accumulates and saves, so the second run starts hot. Budget is VK_EXPERT_GB (auto = device budget minus 1 GB). The engine names its backend in the banner and, on a build without the tier, says so once when COLI_VULKAN is set instead of silently ignoring it (refs #894).

Staged device-local uploads in backend_vulkan.c. Without Resizable BAR the HOST_VISIBLE|DEVICE_LOCAL type is a ~256 MB window and everything past it silently spills to system RAM, so the tier reports resident experts that every access fetches over PCIe. When the host-visible slice is under a quarter of VRAM (or COLI_VK_STAGED=1), resident weights now go to a DEVICE_LOCAL-only arena through a host staging buffer and vkCmdCopyBuffer. Scratches, the KV mirror and readbacks keep their memory types. Queue submits and arena allocation are mutex-protected so the tier's upload thread is safe next to the decode thread.

Validation

Hardware: AMD Radeon RX 580 8 GB (Polaris, gfx803, 256 MB host-visible BAR), Mesa 25.2.8 RADV, i7-7700K, 62 GB RAM, Ubuntu 24.04. First validation of the Vulkan backend on Polaris.

• Backend exactness harness (-DVK_TEST), mapped vs staged: 50/50 maxrel values identical; staged batched int4 matmuls ~4× faster on this card (0.17 vs 0.66 ms/matmul).
• New gate tests/test_qwen36_tier_vk (part of make check; skips without VK=1 or a device): GPU output vs CPU int4 reference maxrel 6.7e-06, misses return no mask bit, residency frozen after warmstart.
• Engine, Qwen3.6-35B-A3B int4-gs64, 15-token prompt, N_NEW=64, 8 threads, greedy, frozen HEAT_FILE on runs 2–3 and the mapped arm, GPU clocks not pinned (no root):

arm	tok/s	TTFT	VRAM-resident	VRAM hit
CPU only	0.63	44.7 s	—	—
Vulkan, cold heat	6.40	1.50 s	3,663 / 10,240	36.1 %
Vulkan, warm heat (2 runs)	6.38 / 5.60	1.48 / 1.50 s	3,655 / 10,240	96.8 %
Vulkan, mapped path (COLI_VK_STAGED=0)	2.44	2.48 s	3,655 / 10,240	96.8 %

Runs 2 and 3 are token-identical; CPU vs GPU output is token-identical on this prompt. Measured on the tree rebased onto current dev (tier commit bb16ab3); an earlier measurement of the same code before the rebase read 7.5 / 4.2 tok/s for the warm and mapped arms, so treat the absolute numbers as ±15 % (clocks unpinned) and the CPU→Vulkan and mapped→staged ratios as the result.

• Tiny-fixture token-exact check 16/16 on both the plain and the VK=1 build (done manually: there is no qwen36-tiny-check target).
• make check green; make qwen36, make qwen36 VK=1, make colibri VK=1 build with 0 warnings from the touched files. CI's Vulkan job now also builds qwen36 VK=1 and runs the tier gate on Lavapipe.

Compatibility

• Default CPU build unchanged and dependency-free (ldd qwen36 links no libvulkan without VK=1). VK=1 is opt-in.
• CUDA tier: allocation-identical (the Vulkan take buffer is compiled out), but one shared-logic fix affects it: the uploader decremented the queue count at dequeue, before the upload ran, so qt_fill_wait could return before the last upload finished while the engine frees the RAM int8 copies right after. Deterministic on the slower staged path (5/5), latent on CUDA. Now an in-flight completion counter; qt_fill_wait returns strictly later.
• Finding I could not resolve: with staged uploads, results computed from a freshly created device-local block differ slightly and non-deterministically unless the block is filled once with vkCmdFillBuffer on creation. The fill value is irrelevant (0x00 and 0xFF measured identical), so it is the first GPU-side touch of the allocation, not a read of filler bytes. The fill is in; the cause is open (no GPU validation layers on my box). Repro: skip the fill and run the harness twice with COLI_VK_STAGED=1.
• Pre-existing, not fixed here (happy to file issues): qt_issue strides G.is_x by di*8*D while it is sized 32*D, so a two-GPU CUDA run with a wide top-k can overrun; qt_shutdown never signals cv_take, so a thread parked in qt_fill_wait would not see th_stop.
• Rebased onto current dev (f58a267). fix(qwen36): the VRAM tier now promotes int8 experts instead of reserving for nothing #1334's int8 expert promotion is kept verbatim and its uploads go through the shim as fmt 1 on both backends; the int8-on-Vulkan path, which neither side had alone, gets its own gate (test_qwen36_tier_vk_int8, runs on Lavapipe in CI). The whole validation above was re-run on the rebased tree.
• Docs: docs/qwen36-cuda-tier.md → docs/qwen36-tier.md (link updated), docs/vulkan.md, docs/ENVIRONMENT.md (COLI_VK_STAGED, VK_EXPERT_GB), CHANGELOG.

🤖 Generated with Claude Code

https://claude.ai/code/session_019BacNGNxAJ1M57UdYE2M3N

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[crichalchemist]

Rebased onto current dev (merge commit 3fb950d, plus one test commit b0503af); GitHub reports the PR mergeable again.

What the merge had to absorb, all inside the backend shim this branch introduces: the fp8 streaming mode, the resident lm_head and DeltaNet projections, automatic placement, and the cudaMalloc-granularity accounting. The shim gained three entry points (be_fp8_set_lut, be_trunk_upload, be_trunk_matmul). On CUDA they map to the existing calls; on Vulkan they refuse with one stderr line and those pieces stay on the CPU path, since backend_vulkan has no matmul over an already-uploaded tensor wired to the tier yet. docs/qwen36-cuda-tier.md is restored as dev has it; docs/qwen36-tier.md now covers only what differs on Vulkan.

The one test change: since d0a382d an expert is charged at allocator granularity, so test_qwen36_tier_vk's hand-computed two-expert budget admitted zero. Same adjustment 823caa8 made to the other tier tests.

Verified locally: the CPU build and the eleven other qwen36 test binaries; and, new since the first push, make qwen36 VK=1 against MoltenVK on macOS 13 x86_64 (2017 iMac, Radeon Pro 580 — the same Polaris10 silicon as the RX 580 rows in the doc, here under Metal rather than RADV), where both Vulkan tier tests run for real (not the skip path) and pass in int4 and int8 mode. Here the Vulkan (Lavapipe) and Qwen3.6 oracle jobs are green; the remaining jobs are still running as I write this.

(Edited: an earlier version of this comment said "Apple GPU"; the device is the AMD Radeon Pro 580.)

Ready for review whenever someone has the time.

[crichalchemist]

Token-exact end to end on real Vulkan hardware, not only the unit tests.

Setup: the ci.yml Qwen3.6 tiny oracle (make_qwen36_tiny.py --ref-mode full, convert_qwen36.py), run through make qwen36 VK=1 on the branch as it now stands (278a267), on a Radeon Pro 580 under MoltenVK (macOS 13 x86_64; the same Polaris10 silicon as the RX 580 rows in docs/qwen36-tier.md, which is this machine booted into Linux). COLI_DENSE_I8=0 SNAP=qwen36_tiny_c COLI_VULKAN=1 VK_EXPERT_GB=auto ./qwen36 8 8 qwen36_tiny/ref_full.json, cap 8 = n_experts so the tier engages.

run	tokens	residency	VRAM hit rate
CPU baseline	16/16	—	—
Vulkan, warmstart, staged uploads (default)	16/16	64/64	100 %
Vulkan, mapped path (COLI_VK_STAGED=0)	16/16	64/64	100 %
Vulkan, int4 conversion (--ebits 4)	16/16	64/64	100 %
Vulkan, QT_NO_WARMSTART=1, 400-token synthetic reference	n/a (ids are synthetic)	64/64 after fill	86 % during the fill

One thing worth knowing for anyone repeating this: on the stock 16-token reference, QT_NO_WARMSTART=1 reports 0 uploads. That is the process finishing in ~8 ms, before the uploader thread lands a single staged copy, not a tier fault; the 400-token row is the same mode given time to fill. The no-warmstart path also never promotes an int8 container on any backend, because the decode path in qwen36.c only calls qt_note for experts that have a packed g4 copy — pre-existing and outside this PR.

The two test_qwen36_tier_vk* unit tests run for real on the same device (not the skip path) and pass in both modes; the CPU build and the eleven other qwen36 test binaries are unchanged.

[crichalchemist]

Pushed 241f7f3: the Vulkan shim now has a gate that runs without a device. tests/qwen36_fake_vulkan.h is the Vulkan twin of qwen36_fake_cuda.h (every coli_vk_* symbol the tier links against, recording what it receives), and test_qwen36_tier_vk_fake drives the shim through it: single device with COLI_GPUS ignored, the VK_EXPERT_GB budget and its 4 GB fallback without VK_EXT_memory_budget, int4-gs64 arriving as fmt 4 / grp 64 at half a byte per element, fill-once under 64 ticks with a far hotter non-resident (no swap, no free), issue/take through ybuf with the weighted accumulation checked lane by lane, a failed take leaving the output alone, and the fp8 and trunk refusals landing on the CPU path without touching the backend.

It is collected by TEST_RULES, so it lands in make check and test-asan on Linux, macOS and Windows alongside the CUDA fake tests; the numerics keep their real-device gate in test_qwen36_tier_vk. Two mutants (a take that always fails; QT_SWAPS=1 on the Vulkan shim) trip one and seven assertions respectively, and it is clean under ASan+UBSan.

Also note #1390, which carries the qt_fill_wait in-flight fix on its own against dev; once it lands I will rebase this branch onto it and drop the copy here.