Skip to content

The decode-graph driver count recorded in 9bc4d7f44 is eight; it is nine, and the ninth predates the record #1179

Description

@localai-bot

9bc4d7f44 (#1161, PR #1167) recorded the hand-rolled decode-graph driver count as
eight in four places. The count is nine, and the ninth was already in the tree
when the record was written.

The miscount

src/vllm/model_executor/models/qwen3_dflash.cpp carries its own decode-graph driver —
g_graph holding a cudaGraphExec (:771 region: void* g_graph = nullptr; // cudaGraphExec (opaque)),
its own d.b.BeginCapture(queue) / d.b.EndCaptureGraph(queue) pair (:1091 region),
its own warm-then-capture state machine (g_state 0 cold / 1 warm / 2 captured), and
DestroyGraph in its destructor. It is a ninth call site by every criterion the record
used for the other eight.

Verified against origin/main at 9bc4d7f44.

Where the wrong count is recorded

  • .agents/specs/sglang-breakable-cuda-graph.md § "The structural finding" — "Eight call
    sites hand-roll capture"
  • .agents/engine-matrix.md, the ENG-CUDAGRAPH-BREAK row — "so the eight hand-rolled
    drivers become one" (also ENG-CUDAGRAPH-DEDUP, "times eight drivers")
  • .agents/roadmap_v1.md track C12 — "hand-rolled eight times"
  • Issue #1163 body — "Eight hand-rolled
    drivers", and "every new model that wants a decode graph writes a ninth one"

Why it matters beyond arithmetic

The #1163 body predicted that the next model wanting a decode graph "writes a ninth one".
That had already happened before the prediction was written. The row's own premise is
therefore stronger than recorded, and the enumeration an implementer would work from is
short by one driver.

Two related asymmetries found in the same sweep, both verified, and both arguing the same
way:

  • StepDevInputs — the persistent-device-input path — appears 41 times in
    src/vllm/model_executor/models/qwen3_5.cpp and 0 times in qwen3_moe.cpp,
    qwen3.cpp, deepseek_v2.cpp, voxtral.cpp and qwen3_dflash.cpp. One driver holds
    the capability; five do not.
  • Because of that, src/vllm/model_executor/models/qwen3.cpp:961-986 declines the decode
    graph outright
    when the async device-token mirror is live, and its own comment records
    the measurement: depth-1, graph ON PASS 78/78 / depth-2, graph OFF PASS 82/82 /
    depth-2, graph ON FAIL, slots 1-3 degenerate. The duplication has already cost a
    shipped model its decode graph on the async serving path.

So ENG-CUDAGRAPH-BREAK is not only a coverage row, as 9bc4d7f44 recorded it. It is
also a correctness row. That framing change is part of this fix.

Scope

Correct the count in all four records, and add the correctness framing above to the
ENG-CUDAGRAPH-BREAK row and to the ## Outcome of
.agents/specs/sglang-breakable-cuda-graph.md. No engine code.

Found while reviewing PR #1177. Owned by ENG-CUDAGRAPH-BREAK.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions