record(ENG-CUDAGRAPH): SGLang's Breakable CUDA Graph, grounded at its pin, and the prefill lever it does not give us (#1161)

.agents/issue-index.md

@@ -347,3 +347,7 @@ rather than merged. `scripts/check-agent-record.py` gates both.
 | [#1150](https://github.com/mudler/vllm.cpp/issues/1150) | `LTX25-TI2VID-RECIPE` | The sigma SHIFT is derived from the target latent on every arm, where six of upstream's seven `LTX2Scheduler.execute` call sites pass NO latent and take `default_number_of_tokens` = `MAX_SHIFT_ANCHOR` = 4096 (`schedulers.py:11,:29,:31`). `grep -rn '\.execute(' packages/ltx-pipelines/src/ltx_pipelines/` at `fd4ded7f` returns seven and that grep is the whole population: only `ti2vid_two_stages_hq.py:267` passes `latent=empty_latent`. This engine passes `target_tokens` at `src/vllm/multimodal/ltx2_video.cpp:3442-3443`, so it mirrors the exception and diverges from the rule. Correct today: `t2a_one_stage` (passes 0 at `src/vllm/model_executor/models/ltx2_t2a.cpp:178`) and `res2s_two_stage`. DIVERGENT: `one_stage` at four version keys (`ti2vid_one_stage.py:207`), `a2vid_two_stage` stage 1 (`a2vid_two_stage.py:226`) and `retake`'s non-distilled arm (`retake.py:287`). Recipes carrying explicit `sigmas` never reach the derivation and are unaffected. Not a rounding difference: at the recipe default geometry the target latent is 6144 tokens, giving `sigma_shift` 2.78 against upstream's 2.05, so every sigma moves. Invisible because the trajectory changes while the frame count, shapes, sample rate and errors do not, and our goldens were captured from this engine so they PIN it rather than detect it. `.agents/specs/ltx25-res2s-loop.md:80-88` saw the HQ/plain split and concluded the divergence was on the plain two-stage arm alone; that is right about HQ and wrong about the blast radius. Found by row `LTX25-TI2VID-RECIPE`, which added the seam — `Ltx2PhaseRecipe::schedule_tokens`, defaulted to today's behaviour so nothing moves — and set it on the one phase it ships. Not fixed in flow because flipping the other three re-samples five shipped, gated arms and rewrites their goldens, which needs its own spec and fresh review. Listed under `## Owed` in [`ltx25-ti2vid-recipe.md`](specs/ltx25-ti2vid-recipe.md) | bug |
 | [#1151](https://github.com/mudler/vllm.cpp/issues/1151) | `LTX25-TI2VID-RECIPE` | The `requires_distilled_lora` refusal still advertised [#1118](https://github.com/mudler/vllm.cpp/issues/1118) as OPEN, and cited `a2vid_two_stage.py`'s line numbers to every other pipeline. #1118 closed at `4ae0f54ab` (row `LTX25-PHASE-LORA`, PR [#1140](https://github.com/mudler/vllm.cpp/pull/1140)), which added `Ltx2PhaseRecipe::loras` and `Ltx2RebindDitLoras`; the message at `src/vllm/multimodal/ltx2_video.cpp:1039-1041` still ended "upstream fuses that adapter into stage 2 ALONE and this engine fuses once at load, so stage 1 sees it too", every clause of which had become false, and the comment above the refusal said the same. `ltx25-phase-lora.md` repaired the REFERENCE-CONDITIONING refusal, which carried the identical claim ~1100 lines away, and named only that one in its port map, so this site and the `ltx2-gen --help` text (`examples/ltx2_gen/main.cpp:210-212`) were both missed. Second defect at the same site: the refusal is deliberately keyed on the FLAG rather than on the kind string so the next recipe inherits it (the comment names #1093 and #1096 as waiting), yet its body interpolated `im.pipeline_kind` into the first sentence and hard-coded a2vid's `:164`, `:114`, `:107` into the rest — so the first arm to inherit it would be told its own name and then a different pipeline's source lines. `--distilled-lora required=True` lives on `default_2_stage_arg_parser` (`utils/args.py:1123`, `:1140-1155`), which all of these pipelines select, and that shared anchor is what the message now cites. Found and fixed IN FLOW by row `LTX25-TI2VID-RECIPE`, the second user of the flag; `tests/vllm/multimodal/test_ltx2_video.cpp` asserted the string `1118` was PRESENT and now asserts it is absent | bug |
 | [#1152](https://github.com/mudler/vllm.cpp/issues/1152) | `LTX25-TI2VID-RECIPE` | `Ltx2PipelineRecipe::allow_request_latents` (`include/vllm/model_executor/models/ltx2_pipeline.h:705`) is WRITTEN by every recipe and READ by nothing — the "a parameter no caller passes" shape AGENTS.md `## Nothing lands dead` names. Measured at `c83b96934`: `grep -rn allow_request_latents src include examples` minus the declaration returns FIVE lines and all five are assignments (`ltx2_pipeline.cpp:1264` false, `:1345` false, `:1473` true, `:1604` false). Positive control, the field declared one line above and set in the same blocks: `allow_request_sigmas` returns its assignments PLUS a real reader at `src/vllm/multimodal/ltx2_video.cpp:3476`, so the grep is well-formed and the absence is the finding. Consequence: `Res2sTwoStageRecipe` carries `true` where every other recipe carries `false` and nothing can tell the difference — no upstream `__call__` among these pipelines takes an initial-latent parameter, so `false` is what the signatures support and the `true` looks like an oversight, but it is unfalsifiable while nothing reads the field. The tests assert the VALUES, so they gate the record against itself and cannot see that nothing consumes it, which is the tautology shape [#911](https://github.com/mudler/vllm.cpp/issues/911) recorded on the anchor checker. Two closes: give it a reader (a refusal on a request supplying a latent to a recipe whose upstream signature has none) or delete it and its assertions — deleting is defensible, since no request surface carries a latent at all so the refusal could never fire either. Found by row `LTX25-TI2VID-RECIPE` while deriving the same field for a sixth recipe; not fixed in flow because both closes touch five landed recipes and one deletes gated assertions. Listed under `## Owed` in [`ltx25-ti2vid-recipe.md`](specs/ltx25-ti2vid-recipe.md) | bug |
+| [#1161](https://github.com/mudler/vllm.cpp/issues/1161) | `ENG-CUDAGRAPH` | SGLang landed **Breakable CUDA Graph (BCG)** and the whole surface was absent from `.agents/sglang-matrix.md`, while `ENG-CUDAGRAPH` carried no verdict on it. Grounded at pin `v0.5.15`/`f63458b5be`: BCG captures one forward as a SEQUENCE of `torch.cuda.CUDAGraph` segments split at eager break points on the attention and recurrent calls, sharing one mempool with weak-ref'd intermediates, with **no `torch.compile`** (`breakable_cuda_graph.py:204-241,244-260,14-23,156-169`; `breakable_cuda_graph_backend.py:14-17`; break points `radix_attention.py:256`, `forward_mla.py:1092`, `radix_linear_attention.py:159`, `nemotron_h.py:1240`, `dsa_indexer.py:2414`). **The public priority claim is narrower than it reads:** vLLM's v1 default is ALREADY `FULL_AND_PIECEWISE`, splitting at `splitting_ops` (`vllm/config/compilation.py:60-63,517,615,630` @ `555967922`), so the capability has been in our benchmark denominator all along — what SGLang landed first is the CONSTRUCTION (piecewise coverage from runtime stream capture plus a decorator, no Dynamo/Inductor/FX), which matters to us only because we have no compiler either. **The load-bearing outcome is a REFUTATION:** prefill graph capture is not a lever on our CUDA gate models, because GB10 2026-07-09 measured steady-state prefill GPU-idle-between-launches at **3.8%** with GPU-busy >96% on both arms, and attributed the 27B prefill gap to **92.5% non-GEMM glue GPU work** with the dominant GEMM the same symbol at the same call count (+0.17%) and attention AHEAD. There are no launch bubbles in our prefill for a graph to collapse, and vLLM's piecewise prefill graph is already inside the denominator we lose to, so it does not explain the gap either. Decode is a separate and already-banked case (conc-64 launch overhead was ~24%, already one `cudaGraphLaunch`/step). FIXED IN FLOW: `SGLANG-BCG` row added under a new "Graph capture and runtime execution" section of `sglang-matrix.md`, the refutation recorded on `ENG-CUDAGRAPH` (which stays `PARTIAL`, because nothing shipped), roadmap track `C12` added, and spec [`sglang-breakable-cuda-graph.md`](specs/sglang-breakable-cuda-graph.md) written to carry the analysis. NOT fixed here and owned by the three issues below: no engine code, no measurement taken, no oracle run, and no advance of the SGLang pin | record |
+| [#1162](https://github.com/mudler/vllm.cpp/issues/1162) | `ENG-CUDAGRAPH-DEDUP` | We instantiate **one `cudaGraphExec` per padded decode bucket, per model**, and `grep -rn "cudaGraphExecUpdate" src include` returns nothing. `src/vt/cuda/cuda_backend.cu:222-232` instantiates a fresh exec per capture; `include/vllm/model_executor/models/decode_graph_sizes.h:32-41` yields 7 buckets at `max_num_seqs=32` and 11 at 64; eight drivers each build their own set. SGLang folds compatible captures onto one executable by hashing graph topology and calling `cudaGraphExecUpdate` on a signature hit (`cuda_graph_dedup_mixin.py:219-242`, logging "captured %d CUDA graphs, deduped to %d execs" at `:358`). Portable to us unchanged, because it is driver-level rather than PyTorch-level. It is a **memory and capture-time** change, NOT a throughput change — a deduped replay launches the same nodes — and it matters because on GB10 unified memory an OOM reboots the box, capture time is startup latency (a recorded gate axis), and bucket count is exactly what widening graph coverage would raise. Owed: the dedup registry behind the `vt` seam, a capture-count/exec-count log line, and a same-binary A/B proving a deduped replay is byte-identical rather than asserting it. Hazard already recorded: capture bakes host source addresses and a clean `compute-sanitizer` run is NOT evidence a capture path is safe (`specs/decode-graph-scratch-uaf-2026-07-18.md`). Spec [`sglang-breakable-cuda-graph.md`](specs/sglang-breakable-cuda-graph.md) `## Owed`. Analysis: [#1161](https://github.com/mudler/vllm.cpp/issues/1161) | perf |
+| [#1163](https://github.com/mudler/vllm.cpp/issues/1163) | `ENG-CUDAGRAPH-BREAK` | Two defects, stated separately. **(1) Capture has no break points:** `src/vllm/v1/worker/gpu/runner.cpp:1338-1341` routes only `pure_decode` batches to a graph, so prefill, mixed batches, and anything whose metadata is computed on the host stay eager for the WHOLE step. There is no way to express "capture this forward except for these three calls", so coverage is a cliff rather than a slope; [#1020](https://github.com/mudler/vllm.cpp/issues/1020) is one instance (a spec verify step whose actual draft depth differs from the configured `k` silently falls out to eager). **(2) Eight hand-rolled drivers:** `Qwen3_5DecodeGraph` (`qwen3_5.h:275`), `Qwen3_5DenseDecodeGraph` (`qwen3_5_dense.h:391`), `Qwen3MoeDecodeGraph` (`qwen3_moe.h:117`), `Qwen3DenseDecodeGraph` (`qwen3.h:243`), `DeepseekV2DecodeGraph` (`deepseek_v2.h:324`), `VoxtralDecodeGraph` (`voxtral.h:126`), plus graph code in `deepseek_v4.cpp` and `laguna.cpp` — each re-deriving capture, bucket padding, persistent-input threading and the pure-decode predicate. AGENTS.md names this shape: a parallel path written by hand instead of a shared seam, and every new model that wants a decode graph writes a ninth one. Mirror vLLM's `CUDAGraphMode.PIECEWISE` boundary (`splitting_ops`, i.e. the attention ops); take the CONSTRUCTION from SGLang BCG, because vLLM gets its split from `torch.compile` and we have no compiler. **Explicitly NOT a throughput row and must not be sold as one** — our prefill has 3.8% host idle and is >96% GPU-busy, and decode already banked its launch-overhead win; the value is coverage and one seam instead of eight. Any speed claim must first name and measure a path that is currently eager AND currently host-bound. Owed: the seam, break-point registration, a reachability mutation (delete the production call site, rerun the focused gate), and bit-exactness vs eager on every migrated model over MORE than one replay. Large and structural; spike first. Spec [`sglang-breakable-cuda-graph.md`](specs/sglang-breakable-cuda-graph.md) `## Owed`. Analysis: [#1161](https://github.com/mudler/vllm.cpp/issues/1161) | feature |
+| [#1164](https://github.com/mudler/vllm.cpp/issues/1164) | `ENG-CUDAGRAPH-DIFFUSION` | Our diffusion path captures **no CUDA graphs at all** — `grep` for capture across `src/vllm/model_executor/models/ltx2*.cpp` returns nothing — and a denoise loop is the ideal capture target (fixed shapes, tens of identical iterations). SGLang enabled BCG on exactly this shape AFTER our pin and measured LTX-2 two-stage H200 e2e 10.75 s → 6.90 s (`d4be483efb`), SANA 1024px e2e -26% (`6c7498113f`), SANA denoise 0.73 → 0.457 s (`56ef810cad`), and Z-Image BCG made bit-exact vs eager (`f5f0c3ee7a`) — cited as DATED UPSTREAM EVENTS, never as pinned evidence, and to be discounted hard because their win is mostly PyTorch host tax our C++ loop does not pay. **BLOCKED, and the blocker is ours:** the premise a denoise-loop graph needs is a denoise loop that RUNS ON THE DEVICE, and three open issues already measured that ours does not. [#1024](https://github.com/mudler/vllm.cpp/issues/1024): a `--device cuda` render stages 35.54 GiB onto the GPU and then does no compute on it — GPU utilization **exactly 0 in 321 of 347 samples**, all 26 non-zero samples inside the staging window, and from t=251 s onward over 17 minutes of 0 while the process holds exactly 1.00 core of 20. [#1007](https://github.com/mudler/vllm.cpp/issues/1007): the video VAE decode has no device arm. [#1087](https://github.com/mudler/vllm.cpp/issues/1087): after [#1041](https://github.com/mudler/vllm.cpp/issues/1041), **57-66% of render wall** is one contiguous single-threaded phase measuring 1731 s and 1732 s across two rungs whose voxel counts differ 2.75x, i.e. resolution-CONSTANT, and [#1010](https://github.com/mudler/vllm.cpp/issues/1010) is open because nothing timestamps a phase boundary so the phase is not even named. A graph collapses host launch dispatch; there is close to no device dispatch here to collapse and the dominant cost is a serial host phase a graph cannot touch, so **capturing now would measure nothing**. Unblock order: [#1010](https://github.com/mudler/vllm.cpp/issues/1010) phase log → [#1087](https://github.com/mudler/vllm.cpp/issues/1087) name and fix the serial phase → [#1024](https://github.com/mudler/vllm.cpp/issues/1024)/[#1007](https://github.com/mudler/vllm.cpp/issues/1007) device residency → THEN measure GPU-busy vs wall. That fourth step is the decision point and it is a MEASUREMENT, not an implementation: if the loop is device-resident and host-dispatch-bound, capture it; if GPU-bound, close it the way [#1161](https://github.com/mudler/vllm.cpp/issues/1161) closed prefill. Gate when it runs: bit-exact PIXELS vs eager over more than one replay, never an exit code (see [#1149](https://github.com/mudler/vllm.cpp/issues/1149)). Spec [`sglang-breakable-cuda-graph.md`](specs/sglang-breakable-cuda-graph.md) `## Owed` | perf |