feat(ENG-CUDAGRAPH-DEDUP): fold captures that share a topology onto one graph executable

[localai-bot]

feat(ENG-CUDAGRAPH-DEDUP): fold captures that share a topology onto one graph executable

Refs #1162 and #1184, deliberately not Closes on either. Row ENG-CUDAGRAPH-DEDUP in
.agents/engine-matrix.md, spec .agents/specs/eng-cudagraph-dedup.md, derived
by the analysis in #1161. #1162 owns every item under the spec's ## Owed --
the device A/B, the executable-count ratio, and the default flip -- and
docs/BENCHMARKS.md records that A/B as PENDING against it. The row stays
ACTIVE. Closing the issue on squash-merge would delete the tracking the
"Nothing lands dead" exception requires while three items are still owed. #1184
stays open for the same reason: the repair below is proven structurally on the
CPU tier and only a device can prove it on a device.

Every capture used to instantiate its own executable and throw the raw graph
away, so a model held one executable per padded decode bucket -- 7 at
max_num_seqs=32, 11 at 64 -- and nine capture drivers each held their own
set. grep -rn "cudaGraphExecUpdate" src include returned nothing. The decode
graphs of two padded batch sizes are usually the same node topology with
different parameters, which is the case cudaGraphExecUpdate exists for.

src/vt/graph_dedup.h is the registry: it keys captures by a structural
signature and re-points one executable instead of instantiating a second.
src/vt/graph_dedup_runtime.h binds it to CUDA and to HIP from one source,
because the two graph APIs differ only by symbol prefix and by two call shapes,
and a second hand-written copy is the parallel path the protocol forbids. Both
accelerator backends route EndCaptureGraph, ReplayGraph and DestroyGraph
through it, dispatching on Owns() rather than a mode flag so a handle minted
before the registry existed is still replayed as the plain executable it is.

The signature is a lookup key and never an authority. Every candidate fold is
probed with the real driver update on a throwaway executable first, and a probe
the driver refuses gives that capture its own executable, so a signature that is
too coarse costs a wasted probe and can never make a replay wrong. That is also
why the probe is transient where SGLang's is persistent: its backend seals the
capture phase and frees the probes, and ours has no capture phase to seal --
the drivers capture lazily, interleaved with replay -- so a persistent probe
would hold two executables per group forever, which is worse than the baseline
this row exists to reduce.

vLLM has no analogue, because its executables come from torch.compile. The
construction is therefore ported from the secondary oracle SGLang at the pinned
f63458b5be, cuda_graph_dedup_mixin.py:27-37,105-179,219-242,258-275,353-358,
with three divergences recorded in the spec's ## Upstream chain.

Not a throughput change

A deduped replay launches the same nodes with the same parameters. The
reportable axes are executable count and capture wall time, and the row must not
be sold as speed. It matters because on GB10 unified memory an out-of-memory
event reboots the box, capture time is startup latency, and bucket count is
exactly what #1163 would raise if it widened coverage.

Gates

• RED first. tests/vt/test_graph_dedup.cpp was written and run before the
  header existed: fatal error: vt/graph_dedup.h: No such file or directory,
  exit 1. Then green, non-vacuous. Now 13 cases / 65 assertions, after the
  review repair below added four cases: two source-behaviour ones run red against
  the unfixed source first (6 failing assertions), one pinning the environment
  polarity, and one gating the loud-failure claim the transitivity note rests on.
• Mutation, 13 of 13 detected (9 at implementation, 4 at review repair). Never fold; never re-point; always re-point;
  trust the signature without probing; keep a freed graph's address; Owns
  claims every pointer; wrong counts in the log line; inverted environment
  polarity; free the shared executable with the first sibling instead of the
  last. Every one printed a non-empty git diff --stat and a successful
  compile before its verdict, and every one restored byte-for-byte -- an earlier
  pass had to be re-run because the file was still untracked, so git diff --stat came back empty and would have hidden a mutation that never applied.
• The suite measures the right thing. The fake device's launch records
  which graph the executable actually reflects, so "the right nodes ran" is an
  observable sequence rather than an assertion about intent, and every case
  replays more than once per shape: the risk of a shared executable is that the
  second visit to a shape is the one that has to re-point it. One case had to
  be added after the fact, because the original sibling-destroy case could not
  see a capture landing on a freed graph's address; the fake now hands that
  storage back and the new case reuses it deliberately.
• CUDA compile. g++ -std=c++20 -fsyntax-only -Wall -Wextra -Werror against
  real CUDA 12.9 runtime headers, exit 0, over a translation unit that also
  mirrors the three call shapes cuda_backend.cu now uses. The cuda-fat-build
  job compiles the real thing on this PR.
• scripts/agent-preflight.sh: All gates green., exit 0.

Review repair

A fresh review returned FAIL. Three findings were code or tests; three are
records, because fixing them would change behaviour the device A/B is measuring
on this branch, or is not code at all.

• A failed cudaGraphInstantiate was silently accepted. Register assigned
  the result and moved on while the runtime returns null on failure. The
  pre-dedup path threw at the capture site with the driver's code; with dedup on
  the same failure minted a valid-looking handle over a null executable, counted
  it live in ExecCount(), and surfaced at the first replay as "graph launch
  failed" -- wrong site, wrong message, driver reason discarded. It now fails at
  the capture site and releases the capture whose ownership it had taken.
• A failed probe drove a null executable. The probe instantiates a second
  executable and can fail on its own; cudaGraphExecUpdate has no defined
  behaviour for a null one. ProbeAccepts now answers "cannot fold", which
  degrades to today's one-executable-per-capture path.
• The environment polarity was unpinned. Deleting && value[1] == '\0'
  left the whole suite green, so "10" enabling a default-off,
  correctness-sensitive path went undetected. The adversarial values are now
  asserted, and the deletion is a detected mutation.
• The fold probed is not the fold replayed (comment and record only).
  Register probes (raws.front(), raw_graph); Replay issues
  (current_raw, entry.raw), and those diverge from a group's third member
  onwards -- so honouring the probe treats cudaGraphExecUpdate compatibility
  as transitive across a group, which no documentation states and nothing here
  asserts. The failure polarity is what makes it survivable: a refusal lands on
  the VT_CHECK in Replay, loudly, never on a silent launch of the previous
  contents. The comment claiming the fold was already probed is corrected to
  say what is actually assumed, and the loud-failure claim it now rests on is
  gated rather than asserted: a new case refuses exactly the
  (currently-reflected, target) pair Register never asks about, so all three
  probes succeed, the group folds to one executable, and the second replay
  throws. Deleting that VT_CHECK makes the replay launch graph 2's nodes under
  graph 3's handle. The stronger fix -- probe current_raw -- is owed, not
  taken: it changes probe behaviour while the A/B is measuring this code.
• src/vt/graph_dedup_runtime.h has no executable coverage (record only).
  The mutation count is a statement about graph_dedup.h. The signature builder
  -- Kahn ordering, topological re-index, sorted edge emission, five node-payload
  cases, the depth-4 bound, five degradation escapes -- is reached by no test on
  any tier, and cuda-fat-build proves only that it compiles. The probe caps the
  blast radius so it can never cause a wrong replay, but one mode is silent: a
  signature unstable for some topology folds nothing while every CPU test stays
  green, with the device-side log line as the only observable. Stated in the spec
  and the matrix row, with the two device-tier tests that would cover it named
  and owed.
• One unsynchronised registry serves every model on a device (record only).
  CudaBackend::dedup_ belongs to a process-singleton per-device backend, so the
  constraint is wider than the "one runner thread" note it carried. Correct
  today; recorded where a future capture-widening row will read it.

Also in this branch: merged up to ab6e6521, resolving the engine-matrix.md
conflict by taking both row sets -- #1177's ENG-CUDAGRAPH-BREAK content
byte-identical, this row's ACTIVE content with #1179's corrected NINE-driver
count carried onto it, and both moves composed in the summary counters. Every
other line of that file is byte-identical to both parents. The spec's Stats()
never existed and is corrected to the shipped CapturedCount() / ExecCount().

#1184: the file was allowed to see the runtime fail, and never consumed the latch

The device gate found that VT_CUDA_GRAPH_DEDUP=1 could not complete one decode
step once a graph was actually captured. 6/6 deterministic on GB10:

vt graph dedup: captured 1 graphs, deduped to 1 execs
[Qwen3DenseDecodeGraph] captured dense decode graph for padded size S=8 (real B=8)
engine-fatal: EngineCore busy loop threw:
  vt cuda: greedy_argmax launch: invalid device function

from a greedy_argmax launch that had SUCCEEDED. The OFF and =0 arms were
clean and byte-identical across 7 runs of the same binary against the same
libraries, which is what makes the asymmetry arm-attributable rather than an
environment artefact.

greedy_argmax was never at fault. The entire safety argument for this row is
that the signature is a lookup key and cudaGraphExecUpdate is the authority --
so a probe the driver REFUSES is normal operation, not an exception -- and the
topology walk has five more escapes of the same kind. A CUDA call that fails
also latches its code in the runtime's sticky per-thread slot, and none of the
twelve fallible calls in src/vt/graph_dedup_runtime.h consumed it: grep -n "cudaGetLastError\|hipGetLastError" on that file returned nothing. The next
unrelated kernel, launched with the ordinary kernel<<<>>>(...); Check(cudaGetLastError()) pattern this tree uses everywhere, then read our
routine refusal and reported it as its own failure.

Every symptom follows from the mechanism and none from the kernel named. It
needs BOTH dedup=1 and a real capture, because with no capture there is no
probe and no latch. CUDA_LAUNCH_BLOCKING=1 does not move it, because the latch
is host-side and synchronous rather than a deferred async error.
cudaGraphLaunch itself returns success, because reading a return value does
not consume the latch.

The repair is one type, not twelve clears. Twelve hand-placed
cudaGetLastError() calls are a fix the thirteenth fallible call silently
misses, and a file whose design is "these calls are allowed to fail" will grow a
thirteenth. So the clear lives in ScopedLatchClear's destructor
(src/vt/graph_dedup_latch.h, new, device-free) and is installed at the
binding's ENTRY POINTS, which are exactly the six GraphDedupOps members the
backends reach through Ops(). Every exit runs it: a plain return, a
degradation escape, and the VT_CHECK unwinding out of Launch.
MakeLatchGuardedOps is the table's only constructor and takes the raw
functions as template arguments, so no raw address reaches a field; a seventh
operation wired anywhere else leaves its field null, and GraphDedupRegistry's
constructor refuses an incomplete table, so the bypass does not merely get
discouraged, it does not construct. One line covers CUDA and HIP because there
is one source: VTGD_FN(GetLastError) resolves to cudaGetLastError or
hipGetLastError.

The fold and probe decision logic is untouched. This is error-state hygiene.

MEDIUM-3: the coverage gap that hid it

src/vt/graph_dedup_runtime.h was reached by no test on any tier --
grep -rn "graph_dedup_runtime\|AppendGraphSignature" tests/ returned nothing --
so its Kahn ordering, topological re-index, sorted edge emission, depth-4 child
bound and degradation escapes were compile-gated only. That is where #1184 sat
through a whole review cycle.

The device-free half moves to src/vt/graph_dedup_signature.h behind an Rt
policy, byte-identical in output, and the new
tests/vt/test_graph_dedup_runtime.cpp drives it with a fake runtime: 13 cases,
51 assertions.

• RED first. With ScopedLatchClear's destructor emptied to the pre-fix
  state, the suite reports 22 failed assertions, including
  CHECK_NOTHROW(NextUnrelatedKernelLaunch()) THREW exception: "greedy_argmax launch: invalid device function" -- the production message reproduced from
  the mechanism alone, with no CUDA in the room. Then green, 13/13.
• Mutation, 7 of 7 detected. The clear does nothing; one table field takes
  the raw address; the Kahn re-index becomes the identity; the edge sort is
  dropped; the depth bound moves 4 to 3; a child graph is noted but never
  walked; the unknown-endpoint escape becomes a continue. Each printed its
  git diff --stat, its compile status and its exit status, and restored
  byte-for-byte. One first attempt failed to build under -Werror and was
  re-run in a compiling form, because a mutation that fails to build reads as a
  passing test.
• What the suite pins: that every entry point clears on every exit path
  including an unwinding one, that a refused fold leaves nothing latched for the
  next caller, that no table field skips the guard, and that the signature walk
  is order-independent, discriminating, bounded and degrading.
• What it cannot pin, stated plainly: a CPU test drives a FAKE runtime. It
  cannot observe the CUDA runtime's real latched-error state, so it proves the
  guard's structure and not that VT_CUDA_GRAPH_DEDUP=1 dies after one replay: the signature walk latches a CUDA error that the next unrelated kernel reports as its own #1184 is gone on a device. The five
  node-payload cases stay device-only behind the policy.
• Both arms compile. No CUDA toolkit was reachable from this session, so a
  scratch header-shape stub instantiated Ops() at CUDART_VERSION 12090 and
  13030 and on the HIP arm under -Werror: all three clean, with a negative
  control proving the instrument can fail. That is a proxy and not the gate --
  the CUDA 13 cudaGraphGetEdges break this file already took was invisible to
  exactly this kind of local check, and only cuda-fat-build reported it.

What the device gate also found, and it bounds this row

vllm-bench on the async path captures NO decode graph at all, because
DenseDecodeGraphForward returns nullopt whenever input.device_token_ids != nullptr -- the #323 mitigation that #1179 re-derived. No capture means no
registration, so dedup engages only under VT_ASYNC_RUNNER=0, which is not the
default and not what a user serves with. This changes nothing about the row's
correctness argument and it does change its worth: on the default path the
executables this row folds are executables that are never instantiated in the
first place. Recorded in the spec rather than sold around; the repair is the
StepDevInputs-shaped one #1179 already owns.

Owed, and named rather than implied

• The device same-binary VT_CUDA_GRAPH_DEDUP off/on A/B proving a deduped
  replay token-identical, and the executable-count ratio. It needs a leased CUDA
  box; dgx:gpu0 was held by another session's oracle build for this whole
  session, and orin:gpu0 has no toolkit. Recorded PENDING in
  docs/BENCHMARKS.md, not omitted.
• Flipping the default on, which is gated on that A/B: a workload alternating
  padded buckets every step pays one update per switch, and shipping a default
  that trades an unmeasured latency for an unmeasured memory saving would be
  the wrong kind of confidence.
• Probing current_raw rather than raws.front(), which retires the
  update-transitivity assumption entirely. Held back because it changes probe
  behaviour mid-measurement; it lands with the A/B rerun. Tracked under One cudaGraphExec per decode bucket per model, and no cudaGraphExecUpdate anywhere: graph executables never dedup #1162.
• Device-tier coverage for the signature builder's remaining half: one test that
  the same trivial graph captured twice signs identically, one that two
  deliberately different topologies sign differently, plus the five node-payload
  cases. The device-free half is now covered by
  tests/vt/test_graph_dedup_runtime.cpp; what is left needs a real
  cudaGraph_t, so it rides with the leased box the A/B already needs. Tracked
  under One cudaGraphExec per decode bucket per model, and no cudaGraphExecUpdate anywhere: graph executables never dedup #1162.
• Re-running the device A/B after the VT_CUDA_GRAPH_DEDUP=1 dies after one replay: the signature walk latches a CUDA error that the next unrelated kernel reports as its own #1184 repair and confirming that
  VT_CUDA_GRAPH_DEDUP=1 now completes a decode step. The CPU suite proves the
  guard's structure over a fake runtime; only a device proves that
  cudaGetLastError clears the real latch. VT_CUDA_GRAPH_DEDUP=1 dies after one replay: the signature walk latches a CUDA error that the next unrelated kernel reports as its own #1184 stays open until that run
  exists.
• Reaching the feature from the DEFAULT serving path at all, which the async
  runner's absent capture blocks today. Tracked under The decode-graph driver count recorded in 9bc4d7f44 is eight; it is nine, and the ninth predates the record #1179.
• The ROCm leg's compile. No ROCm hardware or hipcc is reachable from this
  session and CI has no ROCm job, so the HIP wiring is written against the same
  shared header but is compile-unverified. Tracked under ROCm (AMD GPU) backend #41.

Scope

No driver internals, no bucket selection, no break points (#1163, parallel
owner), no diffusion path (#1164, blocked). docs/ENVIRONMENT.md changes only
because a config key does. scripts/check-gate-commands.py re-pins
RUNNABLE_BASELINE in the same change, which is what its own message demands
when a row enters the runnable population.

What was NOT verified on this head, stated plainly

This lands with two gaps named rather than papered over. Both are owed under #1162
and #1184, which stay open.

The device A/B never ran on the fixed code. dgx:gpu0 was held by another
session's oracle-build130 for over 90 minutes with this job queued at position 1.
The previous run, on the pre-fix head e4ce5571a, is what FOUND #1184: every
dedup=1 cell died after exactly one replay. The repair is therefore unverified on
hardware. What the repair does have is a red-first test reproducing the exact
production string greedy_argmax launch: invalid device function from the mechanism
alone with no CUDA present, 7/7 negative mutations, and a structural design in which
the bypass does not compile.

cuda-fat-build did not report on this head. It passed on e4ce5571a earlier,
before the latch fix added graph_dedup_latch.h and the signature extraction, so the
CUDA leg of the current code is covered only by -Werror stub syntax checks at
CUDART_VERSION 12090 and 13030 plus HIP, each with a negative control proving the
instrument can fail. That job has already caught one real defect on this PR that every
local check passed through, so its silence here is a gap, not a pass.

Why landing anyway is the honest call: VT_CUDA_GRAPH_DEDUP is default OFF, the
unset and =0 arms were byte-identical across 7 runs, and nothing on a default
configuration reaches this code. The "Nothing lands dead" exception is satisfied
literally and was verified by a fresh reviewer. Holding the branch indefinitely on
another session's resource hold would not make any of it more true.

Also bounding this row's value, and measured rather than assumed: the signature
hashes gridDim/blockDim (graph_dedup_runtime.h:121-128), and decode buckets
differ precisely in launch dimensions, so buckets may be structurally unfoldable --
which is the premise #1162 was filed on. The first run logged captured 1 graphs, deduped to 1 execs in every ON cell, a 1:1 ratio with no fold. Separately, the shipped
async serving path captures no decode graph at all, so dedup engages only under
VT_ASYNC_RUNNER=0. Both are recorded in the spec.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]