Skip to content

record(SPEC-DFLASH2): #1456's conclusion is retracted at the artifact, and the 0.8017 denominator inherits it (#1796) - #1800

Merged
localai-bot merged 6 commits into
mainfrom
row/SPEC-DFLASH2-FA-PREMISE
Aug 23, 2026
Merged

record(SPEC-DFLASH2): #1456's conclusion is retracted at the artifact, and the 0.8017 denominator inherits it (#1796)#1800
localai-bot merged 6 commits into
mainfrom
row/SPEC-DFLASH2-FA-PREMISE

Conversation

@localai-bot

@localai-bot localai-bot commented Aug 23, 2026

Copy link
Copy Markdown
Collaborator

#1456 measured a real source build at CUDA_ARCHS=12.0 emitting sm_80 for
_vllm_fa2_C and sm_75 for _vllm_fa3_C, and concluded that vLLM's vendored
flash-attention cannot target sm_12x, so the GB10 oracle has no FLASH_ATTN
denominator. The measurement stands. The conclusion does not, and showing that
needed no lease.
Both staged oracle wheels were opened with zipfile and
walked as fatbinaries on the CPU dev box, and every one of the 76 FA2 and 192
FA3 fatbinaries in each wheel carries a PTX image beside its SASS image. The
first FA2 payload decompresses to .version 9.0 / .target sm_80, which is the
+PTX half of FA2_ARCHS "8.0+PTX" and is the mechanism by which the module CAN
reach sm_121. The artifact establishes a necessary condition, that
forward-JITtable code is shipped; that the JIT then ran is an inference from that
plus a run selecting FLASH_ATTN and generating, which is enough to retract
"cannot target sm_12x" and is not a trace.
cudaErrorUnsupportedPtxVersion is the opposite failure, PTX ISA newer than the
driver. vLLM says the same in its own source:
FlashAttentionBackend.supports_compute_capability returns
capability >= DeviceCapability(8, 0).

The downstream is why this is a change rather than a note. #1456's body records
that the DFlash2 speed gate's denominator would be vLLM pinned to TRITON_ATTN,
by developer decision on 2026-08-20, explicitly because of that conclusion, and
the first DFlash2 speed ratio 0.8016987337853048 was taken against it.
AGENTS.md requires vLLM's production configuration as the denominator, and on
this box vLLM's own auto-selection picks FLASH_ATTN; the gate run's log carries
both branches in one process, cuda.py:426 forcing TRITON_ATTN for the 27B
target and cuda.py:486 auto-selecting FLASH_ATTN for the draft. So if
TRITON_ATTN is the slower backend, which vLLM's priority ordering implies
rather than states and which nothing here measures, the denominator is too low
and the ratio is too high. An
error in our own favour is the one nobody chases, so it is recorded with its
sign.

Nothing is re-measured and no denominator is substituted, because a wave must
not replace a denominator the developer declared. What remains is one lease
running vLLM against itself on this workload with attention_backend=FLASH_ATTN
against TRITON_ATTN, each read back off the built engine, and no GPU lease is
authorised, so it is PENDING.

The same read also settles #1685's three open readings.
vllm/v1/worker/gpu/spec_decode/dflash/utils.py:31-46 sets the draft's
backend=speculative_config.attention_backend unconditionally, so a harness
that sets only the engine backend leaves the draft at None and the draft
re-runs auto-selection; the sibling dspark/utils.py:24-28 and
gemma4/speculator.py:66-89 both carry the target's backend through, and
dspark's comment names the hazard. The DFlash path is the un-defended case, not
an upstream intent.

A fresh review re-derived all twelve cited anchors and confirmed them, with one
enumeration wrong and four notes, and a delta review then caught the repair
shipping a second wrong count. Both are fixed. grep -rln FA_USABLE . --exclude-dir=.git returns three files: .agents/benchmark-record.md, the
DFlash2 spec, and .agents/issue-index.md, where the #1456, #1658 and #1685
rows each quote FA_USABLE=0 as a live constraint. Two of the three are
append-only — the index by rule and gate, benchmark-record.md by its own
opening line, though only as a convention, which is #1373's subject — so the
spec is the only one of the three that can carry the retraction, and ## Owed
O33 says so and names the un-annotated site. What both reviews confirmed
unchanged is that .agents/oracles/ and .agents/upstream-sync.md contain none
of FA_USABLE, FLASH_ATTN or TRITON_ATTN, so no oracle file needs
retracting.

Two departures ride in this branch and are argued rather than assumed. The
appended .agents/issue-index.md row was EDITED by the two repair commits
instead of appended again; scripts/check-issue-index-append-only.py:45-52
diffs merge-base(origin/main, HEAD)..HEAD so it reads the net effect, the row
has never existed on main, and this squash lands one clean append — but a
reader of git log -p on the branch will see the row change shape, which is why
it is said here. And the same implementer who wrote the findings repaired them:
AGENTS.md bars the COORDINATOR from repairing and this is a helper session, but
the same step asks for a FRESH implementer, and these were prose-accuracy
repairs in a records-only change. The freshness that protects correctness held
on the review side — the reviewer re-derived the FA_USABLE enumeration
independently before being told, and re-verified every repaired claim on the new
head.

What is NOT claimed: that an FA2 kernel was traced entering the SM, or that the
forward JIT is free. Construction is proven and coherent output is measured on
both arms; the cost is unmeasured.

Records only, no code. .agents/issue-index.md gains one appended row keyed on
#1796; .agents/specs/dflash2-spec-decode.md gains ## Owed O33 and a fifth
caveat beside the ratio itself. #1456, #1679 and #1685 each received a comment,
and a collapse onto #1685 is recommended there. All three stay open; ending
them is not this branch's call.

Preflight is green apart from test_cpu_x86_llamacpp_floor, which failed with
exit=4 at loadavg 72 on a contended box. That is #618 verbatim and is
unrelated to a records-only change.

Issue: #1796, which stays OPEN because the lease it owes is PENDING. It is
listed under ## Owed O33 of the DFlash2 spec.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [claude-code]

mudler added 6 commits August 23, 2026 16:05
…, and the 0.8017 denominator inherits it (#1796)

#1456 measured a real source build at `CUDA_ARCHS=12.0` emitting `sm_80` for
`_vllm_fa2_C` and `sm_75` for `_vllm_fa3_C`, and concluded that vLLM's vendored
flash-attention cannot target sm_12x, so the GB10 oracle has no `FLASH_ATTN`
denominator. The measurement stands. The conclusion does not, and showing that
needed no lease: both staged oracle wheels were opened with `zipfile` and walked
as fatbinaries on the CPU dev box, and every one of the 76 FA2 and 192 FA3
fatbinaries in each wheel carries a PTX image beside its SASS image. The first
FA2 payload decompresses to `.version 9.0` / `.target sm_80`, which is the
`+PTX` half of `FA2_ARCHS "8.0+PTX"` and is how the module reaches sm_121.
`cudaErrorUnsupportedPtxVersion` is the opposite failure. vLLM says the same in
its own source: `FlashAttentionBackend.supports_compute_capability` returns
`capability >= DeviceCapability(8, 0)`.

The downstream is why this is filed rather than noted. #1456's body records that
the DFlash2 speed gate's denominator would be vLLM pinned to `TRITON_ATTN`, by
developer decision on 2026-08-20, explicitly because of that conclusion, and the
first DFlash2 speed ratio `0.8016987337853048` was taken against it. AGENTS.md
requires vLLM's production configuration as the denominator, and on this box
vLLM's own auto-selection picks `FLASH_ATTN` -- the gate run's log carries both
branches in one process. So if `TRITON_ATTN` is the slower backend, which vLLM's
priority ordering asserts and nothing here measures, the denominator is too low
and the ratio is too high. An error in our own favour is the one nobody chases,
so it is recorded with its sign.

Nothing is re-measured and no denominator is substituted: a wave must not
replace a denominator the developer declared. What remains is one lease running
vLLM against itself on this workload with `attention_backend=FLASH_ATTN` against
`TRITON_ATTN`, each read back off the built engine, and no GPU lease is
authorised, so it is PENDING.

The same read also settles #1685's three open readings. The DFlash draft's
config sets `backend=speculative_config.attention_backend` unconditionally, so a
harness that sets only the engine backend leaves the draft at `None` and the
draft re-runs auto-selection; the sibling dspark and gemma4 speculators both
carry the target's backend through, and dspark's comment names the hazard.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [claude-code]
origin/main advanced to 6756f91 while this records correction was in flight.
The merge is clean and touches nothing this branch edits: the branch changes only
.agents/issue-index.md, which is append-only under merge=union, and
.agents/specs/dflash2-spec-decode.md, which main did not touch.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [claude-code]
…retraction (#1796)

The fresh review re-derived every cited anchor and confirmed all twelve, with
one enumeration wrong and four notes. All are repaired here, and the repairs
land before the row does, so `main` sees one appended index row in its final
form.

The enumeration: `FA_USABLE` also appears in `.agents/issue-index.md` at the
#1456, #1658 and #1685 rows, not only in `.agents/benchmark-record.md` and this
spec. The index is append-only, so those three cannot be edited, which is
exactly why naming them matters; `## Owed` O33 now names all four inheriting
sites and keeps the confirmed part, that `.agents/oracles/` and
`.agents/upstream-sync.md` contain none of `FA_USABLE`, `FLASH_ATTN` or
`TRITON_ATTN`, so no oracle file needs retracting.

The notes. "It is how the module reaches sm_121" is now "the mechanism by which
it CAN reach sm_121", because the artifact establishes a necessary condition and
the JIT having run is an inference; the paragraph that disclaims the kernel
launch no longer contradicts the one above it. "vLLM's own priority ordering
asserts" becomes "implies rather than states", because a preference list is not
a speed claim. #1685's reading 2 is refuted on the runtime half as well as the
constructor half: `flash_attn.py` carries no `fallback` token and
`FlashAttentionImpl.forward` raises `NotImplementedError` rather than degrading.
Reading 1 is wrong AS INTENT, which is the accurate form. A double blank line
before `## Now` is closed.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [claude-code]
…our sites of which three are frozen (#1796)

The delta review closed four of five findings and opened one: the sentence that
repaired a wrong count shipped a second wrong one. "FOUR INHERITING SITES ... AND
THREE OF THEM CANNOT BE EDITED" does not reconcile with its own enumeration,
which names two files plus three index rows, and it undercounts what is frozen.
`.agents/benchmark-record.md:3` self-declares "Append-only forensic record" too,
so BOTH the other files are append-only and this spec is the only one of the
three that can carry the retraction.

O33 now uses the framing the GitHub prose already used and states the measured
thing: `grep -rln FA_USABLE . --exclude-dir=.git` returns three files. It also
names the un-annotated site, `.agents/benchmark-record.md`'s live "#1685 (new,
OPEN)" item, and records that this file's append-only status is a convention
rather than a gate, which is #1373's subject.

TWO DEPARTURES, ARGUED HERE SO THEY STAY WITH THE DIFF. First, the appended
`.agents/issue-index.md` row was EDITED by the two repair commits rather than
appended again. `scripts/check-issue-index-append-only.py:45-52` diffs
`merge-base(origin/main, HEAD)..HEAD`, so it reads the net effect, and the row has
never existed on `main`; the squash lands one clean append. A reader of `git
log -p` on this branch will see the row change shape, and that is why this
paragraph exists. Second, the same implementer who wrote the findings repaired
them. AGENTS.md bars the COORDINATOR from repairing and this session is a helper,
not the coordinator, but the same step asks for a FRESH implementer and these
were prose-accuracy repairs in a records-only change. The control that protects
correctness held: the fresh reviewer re-derived the FA_USABLE enumeration
independently before being told, and re-verified each repaired claim on the new
head.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [claude-code]
…he next append, not by a lock (#1796)

O33 named `.agents/benchmark-record.md`'s live "#1685 (new, OPEN)" item as the
site that quotes `FA_USABLE=0` without a pointer to the retraction, and left
what happens to it unsaid. It says now.

That item is STALE rather than WRONG. It states that `FA-CONSTRAINT.txt` RECORDS
`FA_USABLE=0`, which is still true, and it closes "Unresolved.", which has
stopped being true. Nothing there needs retracting, so the #1783 precedent of an
INSERTED note does not reach it: that precedent exists for a site carrying a
transcription error seven other sites inherited. The retraction rides the next
`SPEC-DFLASH2` entry appended to that file, which is an append and takes no lock.
Annotating in place would take a lock on the one file whose own issue, #1373,
says every appending pull request conflicts, purely to add a forward pointer,
which is the shape AGENTS.md's Records section names.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [claude-code]
…inst a moved main

`main` advanced while this branch's CI ran, and other sessions append to
`.agents/issue-index.md` roughly every twelve minutes, so this branch
re-conflicted on that file alone. No source file conflicted.

The union driver's output was DISCARDED. Locally git honours `merge=union`
(`.gitattributes:7`) and merged without a conflict while RELOCATING this
branch's `#1796` row to interior position 647 of 652. Counts stay correct on a
relocated row; only "is my row the last line" detects it. That is the third time
today this driver has moved a row rather than appended it.

Resolved by AGENTS.md's rule for a concurrent edit to a keyed record: take the
complete target-branch version, then re-apply the scoped edit. Verified three
ways — counts and duplicates, the last row is `#1796`, and sha256 of main's
complete file equals sha256 of this file minus its last line, which proves every
other row byte-identical without inspecting it.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [claude-code]
@localai-bot
localai-bot merged commit 3267bc3 into main Aug 23, 2026
2 of 17 checks passed
@localai-bot
localai-bot deleted the row/SPEC-DFLASH2-FA-PREMISE branch August 23, 2026 22:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants