From 8e4c57967f8e8dd6b61828426217f43f17d83dfe Mon Sep 17 00:00:00 2001 From: Ettore Di Giacinto Date: Thu, 20 Aug 2026 05:05:24 +0000 Subject: [PATCH 01/11] fix(LTX25-RESIDENCY-W0): the anchors are counted by the RENDER, and a swallowed neighbour is now a RED MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three fresh reviews have each broken this gate one level over from the last, and the third one named the shape that made all of it possible: containment, coverage, exclusivity and non-overlap are ALL ratios taken against the sub-scope anchor, so an anchor that moves WITH the leaf defeats four assertions at once. Two assertions that are not ratios close it. RECORD COUNT. Each anchor must be emitted once per unit of work the RENDER counted. `denoise.step` once per `Ltx2ConditioningTrace::dit_evaluations`, `decode.video.chunk` once per `video_decode_chunks` plus the reopen after the last chunk, the audio pair once each. `video_decode_chunks` is new and is incremented in the driver's own streaming sink beside `rendered_frames`; the gate re-derives it from `Ltx2GroupTilesByTemporalSlice` rather than trusting it. Neither counter is derived from the phase table, so no placement of a phase scope can move them. NOTHING BUT AN ANCHOR IS NESTED. A leaf that grows over a NEIGHBOUR does not overlap it and does not break the sum: the neighbour turns `nested`, which excludes it from `sum_leaf_seconds` and from the timeline the exclusivity check walks. That is how the whole PPM writer could be charged to `decode.video` — the leaf W5's lever is measured from — at 99.68% coverage. The gate now names the five anchors and refuses any other nested record, and asserts separately that no `decode.video` leaf encloses `artifacts.frames`. Neither needs a clock. `decode.video` also gains a sub-scope with production events at BOTH ends: `decode.video.vae`, opened in `Ltx2ConvVideoDecodeTiled` around `AccumulateTemporalGroup`. It is the VAE-side sub-scope the spec recorded as owed to W5. And the load, which had no anchor at all: swapping the `load.dit` and `load.prompt_embeds` scope names moved 96% of the load's seconds onto the wrong name with every gate in the file green. `load.dit` is the phase W2 and W3 both act on. An anchor for it would have to open inside `ltx2_loader.cpp`, outside this stage's authority, so what is asserted is the ORDER the driver loads in — which a swapped pair of names breaks — and the case title now says what it checks rather than claiming all fourteen leaves. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code] --- .../demo/ltx25_phase_log_fixture_cpu.json | 2 +- include/vllm/multimodal/ltx2_video.h | 20 ++ .../models/ltx2_video_vae_tiled.cpp | 24 +- src/vllm/multimodal/ltx2_video.cpp | 14 + tests/vllm/multimodal/test_ltx2_video.cpp | 318 +++++++++++++++++- 5 files changed, 363 insertions(+), 15 deletions(-) diff --git a/benchmarks/demo/ltx25_phase_log_fixture_cpu.json b/benchmarks/demo/ltx25_phase_log_fixture_cpu.json index 998be0f90..48f86fc0d 100644 --- a/benchmarks/demo/ltx25_phase_log_fixture_cpu.json +++ b/benchmarks/demo/ltx25_phase_log_fixture_cpu.json @@ -1,6 +1,6 @@ { "_source": "MEASURED 2026-08-20 on mudler-ubuntu-box (Linux 6.8.0-136-generic x86_64, 20 cores, CONTENDED -- load average 31 to 55 while this ran, other sessions compiling throughout). Produced by `build/examples/ltx2-gen`, the shipped example, which is a client of `vllm.h` and nothing else: it calls `vllm_video_engine_load` + `vllm_video_generate` and asks `vllm_video_last_phase_log` for this file's path. Geometry `--frames 9 --width 64 --height 64 --seed 7 --max-phase 0 --device cpu` over the reduced fixture. Built from 3dc2ae98b on row/LTX25-RESIDENCY-W0 with `cmake -DCMAKE_BUILD_TYPE=Release -G Ninja`, gcc 13.3.0. Every byte below `_source`, `_caveat`, `_headline` and `_footnotes` is the render's own output. It REPLACES the artifact taken at e3f46560e, which predates the `denoise.step` and `decode.video.chunk` anchors and therefore could not show them.", - "_caveat": "NOT A BENCHMARK, AND NOT A PROFILE OF ANY SHIPPED CHECKPOINT. The weights are the REDUCED two-block fixture `tests/vllm/multimodal/ltx2_video_fixture.h` writes, not Lightricks' 21.00B DiT, so no duration here describes a real render. The host was contended, and the sharpest evidence for that is in this file's own history: the run taken ONE MINUTE before this one, from the same binary at the same geometry, measured 0.147 s of wall against this run's 4.463 s. Across five such runs the wall has read 0.147 s, 0.158 s, 1.676 s, 4.463 s, 6.138 s and 12.030 s, and the RANK of the two largest phases has reversed between them. Nothing in this file may be quoted as a speed, a ratio between phases, or an ordering. What it supports is the SHAPE of the table -- which phases exist, that they do not overlap, that each carrying phase CONTAINS its own sub-scopes, and that they add up. The `notice` field says the same thing qualitatively and travels with every phase log the library writes.", + "_caveat": "NOT A BENCHMARK, AND NOT A PROFILE OF ANY SHIPPED CHECKPOINT. The weights are the REDUCED two-block fixture `tests/vllm/multimodal/ltx2_video_fixture.h` writes, not Lightricks' 21.00B DiT, so no duration here describes a real render. The host was contended, and the sharpest evidence for that is in this file's own history: the run taken ONE MINUTE before this one, from the same binary at the same geometry, measured 0.147 s of wall against this run's 4.463 s. Across six such runs the wall has read 0.147 s, 0.158 s, 1.676 s, 4.463 s, 6.138 s and 12.030 s, and the RANK of the two largest phases has reversed between them. Nothing in this file may be quoted as a speed, a ratio between phases, or an ordering. What it supports is the SHAPE of the table -- which phases exist, that they do not overlap, that each carrying phase CONTAINS its own sub-scopes, and that they add up. The `notice` field says the same thing qualitatively and travels with every phase log the library writes.", "_headline": "99.94% of an LTX-2.5 render's wall now has a phase name on it, over 33 entries, and each of the three phases that carry the render encloses the sub-scopes that name its work.", "_footnotes": [ "CONTAINMENT, which is what this artifact is re-taken to show. Each carrying leaf encloses its own nested sub-scopes and they cover nearly all of it: denoise.step covers 99.993% of denoise over 8 denoiser evaluations, decode.video.chunk 99.982% of decode.video over 2 leaf records, and decode.audio.mel + decode.audio.vocoder 99.998% of decode.audio. A leaf whose name has been moved onto a neighbour's seconds stops containing its own sub-scopes, which is the one defect a sum cannot see.", diff --git a/include/vllm/multimodal/ltx2_video.h b/include/vllm/multimodal/ltx2_video.h index fdb175542..8610585f5 100644 --- a/include/vllm/multimodal/ltx2_video.h +++ b/include/vllm/multimodal/ltx2_video.h @@ -973,6 +973,26 @@ struct Ltx2ConditioningTrace { // re-noises a keyframe. std::vector video_first_timesteps; + // ── THE VIDEO DECODE, counted by the RENDER (row LTX25-DEVICE-RESIDENCY W0) + // + // How many chunks the streaming video VAE handed back, incremented in the + // driver's own sink beside `rendered_frames`. It is here because W0's phase + // table needs ONE number about the decode that the phase table did not + // produce. + // + // WHY THAT MATTERS AND WHY A COUNTER RATHER THAN A LONGER COMMENT. Every + // assertion W0's containment case makes about `decode.video` — containment, + // coverage, exclusivity, non-overlap — is a RATIO against the leaf, so an + // instrument defect that moves the leaf and its sub-scope TOGETHER satisfies + // all four at once. A count taken by the render is the one quantity such a + // defect cannot move: emit the chunk scope once instead of once per chunk and + // the count disagrees, whatever the clock did. + // + // ONE PER `emit`, so it is the group count `Ltx2GroupTilesByTemporalSlice` + // produces for this request's tiling — which the gate re-derives from that + // function rather than trusting this field. + int64_t video_decode_chunks = 0; + // True only once the `Generate` that produced this conditioning RETURNED. The // trace is filled immediately after the connector and BEFORE the denoise loop, // because that is the only point at which the exact buffers cross-attention diff --git a/src/vllm/model_executor/models/ltx2_video_vae_tiled.cpp b/src/vllm/model_executor/models/ltx2_video_vae_tiled.cpp index a32d79450..f4bc6d627 100644 --- a/src/vllm/model_executor/models/ltx2_video_vae_tiled.cpp +++ b/src/vllm/model_executor/models/ltx2_video_vae_tiled.cpp @@ -41,6 +41,16 @@ #include "vllm/model_executor/models/ltx2_tiling.h" #include "vllm/model_executor/models/ltx2_video_vae.h" +// W0 of LTX25-DEVICE-RESIDENCY (#1010). The one include in this file that is not +// about decoding video, and it is here deliberately: `decode.video` is the leaf +// W5's lever is measured against, and every other assertion the gate makes about +// that leaf is a RATIO taken against the leaf itself. A sub-scope opened in the +// DRIVER, two statements from the leaf's own `Open`, moves with the leaf and +// constrains nothing; a sub-scope opened HERE, around the tile decode the render +// actually spends the seconds in, does not. `render_phase_log.h` is a +// process-wide instrument rather than a multimodal model, and it is the first +// thing under `multimodal/` this directory includes. +#include "vllm/multimodal/render_phase_log.h" #include "vt/dtype.h" namespace vllm { @@ -235,9 +245,17 @@ void Ltx2ConvVideoDecodeTiled(const Ltx2ConvVideoDecoderConfig& config, ChunkBuffer buffer; buffer.Allocate(config.out_channels, curr_stop - curr_start, full_h, full_w); - std::vector curr_weights = - AccumulateTemporalGroup(config, weights, latent, latent_channels, latent_t, latent_h, - latent_w, noise, timestep, group, &buffer, complementary); + // W0 (#1010): THE VIDEO DECODE'S OWN WORK, bounded by production events on + // both ends. One record per temporal group, which is one record per chunk + // the sink is handed, so the count is a quantity the instrument cannot move. + // Nested, so the table's sum does not change. + std::vector curr_weights; + { + const ::vllm::multimodal::phase::Scope vae_phase("decode.video.vae"); + curr_weights = + AccumulateTemporalGroup(config, weights, latent, latent_channels, latent_t, latent_h, + latent_w, noise, timestep, group, &buffer, complementary); + } if (have_previous) { if (previous_stop > curr_start) { diff --git a/src/vllm/multimodal/ltx2_video.cpp b/src/vllm/multimodal/ltx2_video.cpp index 24e824d16..9ea3c305e 100644 --- a/src/vllm/multimodal/ltx2_video.cpp +++ b/src/vllm/multimodal/ltx2_video.cpp @@ -4538,6 +4538,15 @@ VideoResult Ltx2VideoEngine::Generate(const VideoGenParams& gen) { // statement. A `decode.video` leaf that closes before its chunk arrives, or // that is re-labelled after one, no longer contains the chunk it produced. // Nested, so the sum does not move. + // + // ITS OPEN IS STILL THE STATEMENT NEXT DOOR, and the third fresh review showed + // what that costs: move the two `Close`s below to after the PPM write and the + // leaf AND its anchor grow together, so containment and coverage both hold + // while the whole writer is charged to `decode.video`. Two things close it. + // `decode.video.vae` is opened inside `Ltx2ConvVideoDecodeTiled` around + // `AccumulateTemporalGroup`, so `decode.video` has one sub-scope whose ends + // are both production events; and the gate asserts that nothing but an anchor + // is emitted NESTED, which is what a swallowed `artifacts.frames` becomes. size_t chunk_handle = phase::PhaseLog::Instance().Open("decode.video.chunk", /*span=*/false); Ltx2VideoDecodeStreaming( @@ -4568,6 +4577,11 @@ VideoResult Ltx2VideoEngine::Generate(const VideoGenParams& gen) { } rendered_frames += chunk.frames.frames; rendered_channels = chunk.frames.channels; + // COUNTED BY THE RENDER, not by the instrument (#1010, third fresh + // review). W0's containment case needs one number about this decode that + // no phase-scope placement can move; this is it, and it sits beside the + // frame count for the same reason that one does. + im.trace.video_decode_chunks += 1; write_phase.Close(); decode_handle = phase::PhaseLog::Instance().Open("decode.video", /*span=*/false); chunk_handle = phase::PhaseLog::Instance().Open("decode.video.chunk", /*span=*/false); diff --git a/tests/vllm/multimodal/test_ltx2_video.cpp b/tests/vllm/multimodal/test_ltx2_video.cpp index fb8d73b64..2f35d90d2 100644 --- a/tests/vllm/multimodal/test_ltx2_video.cpp +++ b/tests/vllm/multimodal/test_ltx2_video.cpp @@ -2909,8 +2909,32 @@ TEST_CASE("ltx2 video: a render through the ABI emits a phase table that SUMS to // whose honest denoise was 73.4%. A gate blind to that is a gate this campaign // cannot rank levers with, because W1 ranks them off this very table. // -// So each carrying phase now carries an ANCHOR, and this case asserts four -// things about it: +// AND A THIRD FRESH REVIEW BROKE IT AGAIN, one level over from that. All four +// assertions below are RATIOS taken against the sub-scope anchor, so an anchor +// that moves WITH the leaf defeats all four at once. Two mutations, both +// building clean, both exit 0 on both cases: +// +// M12 emits `denoise.step` for `step_index == 0` only, closes `denoise` after +// step 0 and opens `phase.finish` over steps 1-7. **exit 0, 106/106 and +// 350/350 assertions**, over a table reporting `phase.finish` at 28.73% +// against a `denoise` of 6.11% on a run whose honest denoise was 39.3%. +// That is M7's defect again, on M7's phase, through the gate M7 produced. +// M11 moves the two `Close`s in the driver's chunk sink to after the PPM +// write. **exit 0, 124/124 and 422/422**, coverage still 99.68%, with the +// whole PPM writer charged to `decode.video` — the leaf W5's lever is +// measured from — because a swallowed neighbour turns `nested`, leaves the +// sum, and overlaps nothing. +// +// So the case now asserts SIX things, and the first and the sixth are the ones +// that are not ratios: +// +// 0. RECORD COUNT the anchor runs once per unit of work the RENDER counted. +// `denoise.step` once per `Ltx2ConditioningTrace:: +// dit_evaluations`, `decode.video.chunk` once per +// `video_decode_chunks` plus the reopen after the last one, +// the audio pair once each. Neither counter is derived from +// the phase table, so no placement of a scope can move them. +// This is what M12 fails. // // 1. CONTAINMENT every sub-scope interval lies inside one of the leaf's own // records. `denoise.step` wraps the denoiser evaluation inside @@ -2931,6 +2955,16 @@ TEST_CASE("ltx2 video: a render through the ABI emits a phase table that SUMS to // is the assertion that catches the transfer directly: M7's // `phase.finish` opens in the middle of the denoise window. // 4. NON-OVERLAP the carrying leaves and the writer never overlap each other. +// 5. NOTHING BUT AN ANCHOR IS NESTED, plus the writer is never enclosed by a +// `decode.video` leaf. A leaf that grows over a NEIGHBOUR +// does not overlap it and does not break the sum — the +// neighbour turns `nested` and leaves the timeline — so this +// is the assertion M11 fails, and it needs no clock. +// `decode.video` also gains `decode.video.vae`, opened +// inside `Ltx2ConvVideoDecodeTiled` around +// `AccumulateTemporalGroup`, so that leaf has one sub-scope +// whose ends are both production events rather than +// statements adjacent to its own. // // WHAT WAS TRIED FIRST AND REJECTED, MEASURED RATHER THAN ARGUED. The obvious // repair is a differential: render at two frame counts and require the phases @@ -3009,6 +3043,12 @@ struct Carrying { std::vector parts; // the nested sub-scopes sitting on the work double min_coverage = 0.0; // of the leaf's own seconds std::vector partners; // leaves the driver genuinely interleaves with + // HOW MANY RECORDS OF EACH PART, from a number the RENDER produced. One entry + // per name in `parts`. This is the only assertion here that is not a ratio + // against the leaf, and it is why the other four are worth making — see the + // note above this case. + std::vector part_counts; + std::string count_source; // where those numbers came from, for the message }; // The four assertions above, for one carrying phase. A free function rather than @@ -3027,6 +3067,33 @@ void CheckCarryingPhase(const nlohmann::json& table, const Carrying& c) { } REQUIRE(leaf_seconds > 0.0); + // (0) THE RECORD COUNT, and it is FIRST because it is the only one of the five + // that is not a ratio against `leaf_seconds`. See the note above this case: + // containment, coverage, exclusivity and non-overlap all measure the anchor + // AGAINST the leaf, so an anchor that moves WITH the leaf satisfies all four + // at once, and mutation M12 did exactly that — emit `denoise.step` for step 0 + // only, close `denoise` there, open `phase.finish` over the remaining seven + // steps, and both cases in this file report exit 0 over a table that charges + // 82% of the largest phase to its neighbour. + // + // The counts come from `Ltx2ConditioningTrace`, which the RENDER maintains: an + // evaluation increments `dit_evaluations` inside the shared `Evaluate` lambda + // and the driver's own sink increments `video_decode_chunks` beside + // `rendered_frames`. Neither is derived from the phase table, so no placement + // of a phase scope can move them, and an anchor that runs a different number + // of times than the work it claims to wrap is caught whatever the clock did. + REQUIRE(c.part_counts.size() == c.parts.size()); + for (size_t i = 0; i < c.parts.size(); ++i) { + const int64_t emitted = static_cast(RecordsNamed(table, c.parts[i]).size()); + INFO("sub-scope = " << c.parts[i]); + CHECK_MESSAGE(emitted == c.part_counts[i], + "'" << c.parts[i] << "' was emitted " << emitted << " time(s) and the render " + << "did the work " << c.part_counts[i] << " time(s) (" << c.count_source + << "). An anchor that does not run once per unit of work it names is not " + "anchoring the name to the work — it is measuring whichever unit it " + "happened to be opened around"); + } + // (1) CONTAINMENT, and the sub-scopes' own intervals. std::vector> subs; for (const std::string& part : c.parts) { @@ -3103,9 +3170,75 @@ void CheckCarryingPhase(const nlohmann::json& table, const Carrying& c) { } } +// NOTHING BUT AN ANCHOR IS NESTED, and this is the assertion that closes the +// GROWING direction the four above cannot see. +// +// A record is `nested` when it opened while another leaf was already open +// (`render_phase_log.cpp`, `PhaseLog::Open`), and a nested record is dropped from +// `sum_leaf_seconds`. So a carrying leaf that grows over one of its NEIGHBOURS +// does not overlap anything and does not break any sum — the neighbour simply +// turns nested and leaves the timeline. Mutation M11 is exactly that: move the +// two `Close`s in the driver's chunk sink to after the PPM write and the whole +// writer is charged to `decode.video`, which is the leaf W5's lever is measured +// from. Containment held, coverage held at 99.68%, non-overlap held because the +// leaves stayed disjoint, and exclusivity held because `artifacts.frames` is a +// DECLARED PARTNER of `decode.video` and is skipped by name. +// +// The list is the anchors and nothing else, so the assertion needs no clock and +// no threshold: any other name appearing nested means a leaf was left open +// across work it does not name. The driver's own comment above the sink +// (`ltx2_video.cpp`, "Folding the writes into `decode.video` would charge W5's +// lever with the cost of a `write(2)`; nesting them would take the writer out of +// the sum entirely") states both halves of this as a requirement; before this +// line no gate held it. +void CheckOnlyAnchorsAreNested(const nlohmann::json& table, + const std::vector& anchors) { + for (const nlohmann::json& e : table["phases"]) { + if (!e.value("nested", false)) continue; + const std::string name = e["name"].get(); + INFO("nested record = " << name); + CHECK_MESSAGE(std::find(anchors.begin(), anchors.end(), name) != anchors.end(), + "'" << name << "' is emitted NESTED, and it is not one of this render's " + "anchors. A nested record is excluded from `sum_leaf_seconds`, so a leaf " + "that grew across this one moved its seconds onto itself and took it out " + "of the table at the same time — with no overlap, no gap and no change to " + "what the phases add up to"); + } +} + +// ...and the converse for the pair the driver INTERLEAVES. `artifacts.frames` +// alternates with `decode.video` rather than nesting inside it, which is what +// keeps the `write(2)` off W5's lever. Asserted over the INTERVALS as well as +// over the `nested` flag, so a change in what the emitter means by `nested` +// cannot quietly retire it. +void CheckWriterIsBesideTheDecode(const nlohmann::json& table) { + const std::vector writes = RecordsNamed(table, "artifacts.frames"); + REQUIRE_MESSAGE(!writes.empty(), "the render wrote frames and named no 'artifacts.frames' leaf"); + const std::vector decodes = RecordsNamed(table, "decode.video"); + REQUIRE(!decodes.empty()); + for (const nlohmann::json& w : writes) { + const double ws = w["start_seconds"].get(); + const double we = w["end_seconds"].get(); + INFO("artifacts.frames = [" << ws << ", " << we << "]"); + CHECK_FALSE_MESSAGE(w.value("nested", false), + "the PPM writer is NESTED, so its seconds are charged to whatever leaf " + "encloses it and dropped from the sum"); + for (const nlohmann::json& d : decodes) { + const double ds = d["start_seconds"].get(); + const double de = d["end_seconds"].get(); + const bool outside = we <= ds + 1e-9 || ws >= de - 1e-9; + CHECK_MESSAGE(outside, + "a 'decode.video' leaf runs [" << ds << ", " << de << "] and ENCLOSES the " + << "writer at [" << ws << ", " << we << "]. W5 ranks this campaign's " + << "levers off the `decode.video` number and it would be reading the " + << "cost of a write(2)"); + } + } +} + } // namespace -TEST_CASE("ltx2 video: each named phase CONTAINS the work it is named after") { +TEST_CASE("ltx2 video: the three carrying phases contain their work and the load keeps its order") { Workspace ws; // `max_phase = 0` is a LOAD extra and is what keeps this on one recipe phase: // the default `distilled_two_stage` kind would need the latent spatial @@ -3115,6 +3248,9 @@ TEST_CASE("ltx2 video: each named phase CONTAINS the work it is named after") { const std::unique_ptr engine = vllm::multimodal::LoadVideoEngine(mp); REQUIRE(engine != nullptr); + vllm::multimodal::Ltx2VideoEngine* ltx2 = + dynamic_cast(engine.get()); + REQUIRE(ltx2 != nullptr); const vllm::multimodal::VideoResult result = engine->Generate(FixtureGen(ws.root + "/attribution")); @@ -3122,6 +3258,35 @@ TEST_CASE("ltx2 video: each named phase CONTAINS the work it is named after") { REQUIRE_MESSAGE(!result.phase_log_path.empty(), "the render wrote no phase table"); const nlohmann::json table = nlohmann::json::parse(ReadAll(result.phase_log_path)); + // ── WHAT THE RENDER COUNTED, which is not what the instrument counted ────── + // + // Two numbers, both maintained by the render itself. `dit_evaluations` is + // incremented inside the shared `Evaluate` lambda every sampler arm goes + // through; `video_decode_chunks` is incremented in the driver's streaming sink + // beside `rendered_frames`. They are the anchors' denominators below. + const vllm::multimodal::Ltx2ConditioningTrace trace = ltx2->last_conditioning(); + REQUIRE(trace.completed); + REQUIRE(trace.dit_evaluations > 0); + REQUIRE(trace.video_decode_chunks > 0); + + // AND THE CHUNK COUNT IS RE-DERIVED rather than taken on faith, in the shape + // the multi-chunk case above already uses: the tiling algebra the decoder runs + // is public, so the group count for this request is computable here from the + // fixture's own factors. A counter that was incremented in the wrong place + // would agree with the phase table and disagree with this. + { + const vllm::Ltx2ScaleFactors factors{8, 32, 32}; + const vllm::Ltx2TileSizeConfig layout = vllm::Ltx2AutoTileSizeConfig(32, 32, factors); + const int64_t latent_t = (result.frame_count - 1) / factors.time + 1; + const std::vector tiles = + vllm::Ltx2CreateTiles(latent_t, 1, 1, layout, factors); + const int64_t groups = + static_cast(vllm::Ltx2GroupTilesByTemporalSlice(tiles).size()); + CHECK_MESSAGE(trace.video_decode_chunks == groups, + "the render counted " << trace.video_decode_chunks << " decoded chunk(s) and " + << "this request's tiling produces " << groups << " temporal group(s)"); + } + // (1)-(3) THE THREE PHASES THAT CARRY THIS RENDER, each with its anchor. // // The coverage thresholds are NOT round numbers and are not the same, because @@ -3134,11 +3299,23 @@ TEST_CASE("ltx2 video: each named phase CONTAINS the work it is named after") { // with was a HOLE the second review named: at 0.90 `decode.audio` could // open 11% early and swallow 0.13 s, which is 87% of this render's entire // `decode.video`, while passing everything in this file. - // * `denoise.step` covers the denoiser EVALUATIONS, measured at 99.67% over - // 8 of them. The uncovered part is real work — the post-process and the - // Euler or res_2s step, which no anchor wraps — PLUS two instrument - // boundaries per evaluation, and a preempted `/proc/self/statm` read at one - // of those costs a percent of a 40 ms leaf on this fixture. 0.95. + // * `denoise.step` covers the denoiser EVALUATIONS, measured at 99.37% and + // 99.67% over 8 of them. The uncovered part is real work — the post-process + // and the Euler or res_2s step, which no anchor wraps — PLUS two instrument + // boundaries per evaluation. 0.95. + // + // THE MARGIN IS A FALSE-RED RISK AND THIS COMMENT USED TO ARGUE THE + // OPPOSITE. It said the uncovered part "costs a percent of a 40 ms leaf", + // which reads as though the overhead scales with the leaf. It does not: the + // 16 boundary samples cost the same wall whether the denoise is 20 ms or 20 + // minutes, so the RATIO gets worse exactly as the hardware gets faster. A + // third fresh review measured 99.228% on a quiet box at a 20 ms leaf — an + // uncovered 0.154 ms — which puts the crossing point at a denoise leaf of + // about 3.1 ms. Below that this threshold reds a healthy tree. It is a + // FALSE RED and never a false pass, and it is left at 0.95 rather than + // loosened on an argument nobody has measured: the (0) record count is what + // now ties this name to its work, and a red here has a diagnosis printed + // beside it in the MESSAGE above. // * `decode.video.chunk` runs from the leaf's own open to the moment the // decoder hands a chunk back, so the only uncovered part is TWO instrument // boundaries — measured 99.44% — and that is why its threshold is the @@ -3149,13 +3326,78 @@ TEST_CASE("ltx2 video: each named phase CONTAINS the work it is named after") { // go unnamed, which is three orders of magnitude below the 0.13 s the same // 0.90 permitted on the audio decode: the fraction is the same and the // stake is not, which is why these three numbers are not one number. + // + // AND EACH CARRIES ITS RECORD COUNT, which is assertion (0) and the one that + // is not a ratio. `denoise.step` runs once per denoiser evaluation. + // `decode.video.chunk` opens with the leaf and reopens after every chunk the + // sink is handed, so it emits one more record than there are chunks — the last + // one is the empty window between the final chunk and the end of the decode. + // The audio decode is exactly one mel pass and one vocoder pass. const std::vector carrying = { - {"denoise", {"denoise.step"}, 0.95, {}}, - {"decode.video", {"decode.video.chunk"}, 0.90, {"artifacts.frames"}}, - {"decode.audio", {"decode.audio.mel", "decode.audio.vocoder"}, 0.99, {}}, + {"denoise", + {"denoise.step"}, + 0.95, + {}, + {trace.dit_evaluations}, + "Ltx2ConditioningTrace::dit_evaluations"}, + {"decode.video", + {"decode.video.chunk"}, + 0.90, + {"artifacts.frames"}, + {trace.video_decode_chunks + 1}, + "Ltx2ConditioningTrace::video_decode_chunks + 1, the reopen after the last chunk"}, + {"decode.audio", + {"decode.audio.mel", "decode.audio.vocoder"}, + 0.99, + {}, + {1, 1}, + "the two calls #1010 names, once each per render"}, }; for (const Carrying& c : carrying) CheckCarryingPhase(table, c); + // (3b) THE VAE'S OWN SUB-SCOPE, which is where the video decode spends its + // seconds and the reason `decode.video.chunk` is not the whole answer. + // + // `decode.video.chunk` opens two statements after `Open("decode.video")` and + // closes one statement before `Close("decode.video")`, so containment and + // coverage over it hold BY CONSTRUCTION and cannot constrain the leaf in the + // growing direction — which is what M11 exploited. `decode.video.vae` is + // opened inside `Ltx2ConvVideoDecodeTiled` around `AccumulateTemporalGroup`, + // i.e. around the tile decode itself, so both of its ends are production + // events. One record per temporal group, which is one per chunk, and each one + // inside the chunk window that produced it. + { + const std::vector vae = RecordsNamed(table, "decode.video.vae"); + CHECK_MESSAGE(static_cast(vae.size()) == trace.video_decode_chunks, + "the tiled decode emitted " << vae.size() << " 'decode.video.vae' record(s) " + << "against " << trace.video_decode_chunks << " chunk(s) the render " + << "counted"); + const std::vector chunks = RecordsNamed(table, "decode.video.chunk"); + for (const nlohmann::json& v : vae) { + CHECK_MESSAGE(v.value("nested", false), + "'decode.video.vae' is not marked nested, so the tile decode is being SUMMED " + "as well as the leaf that contains it"); + const double vs = v["start_seconds"].get(); + const double ve = v["end_seconds"].get(); + bool inside = false; + for (const nlohmann::json& ch : chunks) { + if (vs >= ch["start_seconds"].get() - 1e-9 && + ve <= ch["end_seconds"].get() + 1e-9) { + inside = true; + } + } + CHECK_MESSAGE(inside, "a 'decode.video.vae' record runs [" << vs << ", " << ve + << "] and no 'decode.video.chunk' window encloses it"); + } + } + + // (3c) AND NOTHING BUT AN ANCHOR IS NESTED. The assertion that sees a leaf + // swallow a NEIGHBOUR, which the four above cannot: see the note on + // `CheckOnlyAnchorsAreNested`. + CheckOnlyAnchorsAreNested(table, {"denoise.step", "decode.video.chunk", "decode.video.vae", + "decode.audio.mel", "decode.audio.vocoder"}); + CheckWriterIsBesideTheDecode(table); + // (4) THE FLOOR, read as "this name is not detached" and nothing more. See the // note above this case for why it is not tightened toward the measured share. const std::map leaves = LeafSecondsByName(table); @@ -3198,6 +3440,60 @@ TEST_CASE("ltx2 video: each named phase CONTAINS the work it is named after") { << spans[i - 1].second << "] and [" << spans[i].first << ", " << spans[i].second << "]"); } + + // (6) THE LOAD LEAVES SIT IN THE DRIVER'S OWN ORDER, and this is the weakest + // assertion in the case as well as the only one the load has. + // + // WHY IT EXISTS. The three assertions above cover the three phases that carry + // a RENDER. They say nothing about the six the LOAD is made of, and a third + // fresh review showed what that costs: swap the `load.dit` and + // `load.prompt_embeds` scope NAMES in the driver — two lines — and every gate + // in this file stays green while 96% of the load's seconds move onto the wrong + // name. On the shipped 21B that name holds about seven and a half minutes of + // DiT staging, and it is the phase W2 and W3 both act on. + // + // WHY IT IS ONLY AN ORDER. An anchor for `load.dit` would have to open inside + // `Ltx2LoadDitFromSafetensors` / `Ltx2StreamDitToDevice` + // (`ltx2_loader.cpp`), which this stage's authority does not reach; the driver + // has exactly one statement between the scope's open and its close, so a + // sub-scope placed HERE would be the adjacent-statement anchor whose + // circularity M11 and M12 already demonstrated. The order is what remains + // checkable from the table alone: the driver loads the DiT, then the video + // VAE, then the audio VAE, then the upsampler, then the text encoder, then the + // prompt embeds, and a swapped pair of names shows up here as a swapped pair + // of start times. What is still NOT proven is that each of these names covers + // the work it is named after; `### Owed out of W0` in the row's spec lists + // every leaf in that state, and W2 and W3 must read it before they act on + // `load.dit`. + { + const std::vector load_order = {"load.dit", "load.video_vae", + "load.audio_vae", "load.upsampler", + "load.text_encoder", "load.prompt_embeds"}; + double previous = -1.0; + std::string previous_name; + int64_t seen = 0; + for (const std::string& name : load_order) { + const std::vector found = RecordsNamed(table, name); + // A load phase that did not run emits nothing; this fixture carries no + // upsampler and no text encoder. + if (found.empty()) continue; + CHECK_MESSAGE(found.size() == 1u, + "'" << name << "' was emitted " << found.size() + << " times and the driver opens it once per load"); + const double start = found.front()["start_seconds"].get(); + CHECK_MESSAGE(start >= previous, + "'" << name << "' starts at " << start << " and '" << previous_name + << "' starts at " << previous << ", so the table reports the load in an " + "order the driver does not run it in. Two scope names that were " + "swapped move their seconds onto each other and change nothing else " + "in this file"); + previous = start; + previous_name = name; + ++seen; + } + REQUIRE_MESSAGE(seen >= 4, "this fixture load names only " << seen + << " load phases; the order it checks is vacuous"); + } } // ─── the CONSOLE copy is not a passenger of the FILE write ────────────── From 940fd2e6cd658175ceb05c40fb84e5a3051aa918 Mon Sep 17 00:00:00 2001 From: Ettore Di Giacinto Date: Thu, 20 Aug 2026 05:14:33 +0000 Subject: [PATCH 02/11] fix(LTX25-RESIDENCY-W0): the PPM writer gets its own anchor, because emptying its leaf was free MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The third review's M11 leaves the `write(2)`s inside `decode.video` and emits `artifacts.frames` covering nothing — four microseconds for nine PPM files. The leaves stay disjoint, nothing nests, the sum does not move, and every interval assertion in the file holds, including the ones added an hour ago: the writer had no anchor, so there was nothing to say its name had stopped covering its work. `artifacts.frames.ppm` wraps the `WriteFileBytes` loop, so it moves with the WRITES rather than with the scope beside it, and `artifacts.frames` joins the carrying phases with its record count taken from `video_decode_chunks`. Its coverage threshold is 0.50 and deliberately loose: the leaf is 0.2 ms on this fixture and carries its own scope boundaries, so what binds here is the containment and the count, not the fraction. Measured 97.97% honest. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code] --- src/vllm/multimodal/ltx2_video.cpp | 13 +++++++++++++ tests/vllm/multimodal/test_ltx2_video.cpp | 21 ++++++++++++++++++++- 2 files changed, 33 insertions(+), 1 deletion(-) diff --git a/src/vllm/multimodal/ltx2_video.cpp b/src/vllm/multimodal/ltx2_video.cpp index 9ea3c305e..ef07fd4cc 100644 --- a/src/vllm/multimodal/ltx2_video.cpp +++ b/src/vllm/multimodal/ltx2_video.cpp @@ -4562,6 +4562,18 @@ VideoResult Ltx2VideoEngine::Generate(const VideoGenParams& gen) { shape.t = chunk.frames.frames; shape.h = chunk.frames.height; shape.w = chunk.frames.width; + // W0 repair (#1010, third fresh review): THE WRITER'S OWN ANCHOR, bound + // to the `WriteFileBytes` loop rather than to the scope beside it. + // + // Without it the writer is held by nothing. Empty this leaf and leave + // the `write(2)`s inside `decode.video` — which is what happens if the + // two `Close`s above move below the loop — and the table charges W5's + // lever with the cost of the writes while `artifacts.frames` reports + // four microseconds for nine files. Every interval assertion still + // holds, because the leaves stay disjoint and nothing nests. This scope + // moves with the WRITES, so a leaf that stops covering them stops + // containing it. Nested, so the sum does not move. + phase::Scope ppm_phase("artifacts.frames.ppm"); for (int64_t f = 0; f < chunk.frames.frames; ++f) { char name[64]; // The GLOBAL frame index, which the chunk carries so the writer does not @@ -4575,6 +4587,7 @@ VideoResult Ltx2VideoEngine::Generate(const VideoGenParams& gen) { WriteFileBytes(JoinPath(gen.output_dir, name), MiniMaxH3WritePpmFrame(chunk.frames.data, shape, f)); } + ppm_phase.Close(); rendered_frames += chunk.frames.frames; rendered_channels = chunk.frames.channels; // COUNTED BY THE RENDER, not by the instrument (#1010, third fresh diff --git a/tests/vllm/multimodal/test_ltx2_video.cpp b/tests/vllm/multimodal/test_ltx2_video.cpp index 2f35d90d2..c900afb19 100644 --- a/tests/vllm/multimodal/test_ltx2_video.cpp +++ b/tests/vllm/multimodal/test_ltx2_video.cpp @@ -3352,6 +3352,24 @@ TEST_CASE("ltx2 video: the three carrying phases contain their work and the load {}, {1, 1}, "the two calls #1010 names, once each per render"}, + // AND THE WRITER, which shipped with no anchor at all and is the leaf the + // `decode.video` number is protected by. `artifacts.frames.ppm` wraps the + // `WriteFileBytes` loop, so it moves with the WRITES: a mutation that + // empties `artifacts.frames` and leaves the `write(2)`s inside + // `decode.video` — the reviewer's M11, which reported four microseconds + // for nine PPM files — no longer has a leaf that contains this record. + // + // 0.50 is the threshold and it is loose ON PURPOSE. This leaf is about + // 0.2 ms on the fixture and holds nine scope boundaries of its own, so the + // uncovered part is instrument cost against a sub-millisecond leaf, which + // is the regime the `decode.video` note above already explains. What binds + // here is the containment and the count, not the fraction. + {"artifacts.frames", + {"artifacts.frames.ppm"}, + 0.50, + {"decode.video"}, + {trace.video_decode_chunks}, + "Ltx2ConditioningTrace::video_decode_chunks, one write callback per chunk"}, }; for (const Carrying& c : carrying) CheckCarryingPhase(table, c); @@ -3395,7 +3413,8 @@ TEST_CASE("ltx2 video: the three carrying phases contain their work and the load // swallow a NEIGHBOUR, which the four above cannot: see the note on // `CheckOnlyAnchorsAreNested`. CheckOnlyAnchorsAreNested(table, {"denoise.step", "decode.video.chunk", "decode.video.vae", - "decode.audio.mel", "decode.audio.vocoder"}); + "decode.audio.mel", "decode.audio.vocoder", + "artifacts.frames.ppm"}); CheckWriterIsBesideTheDecode(table); // (4) THE FLOOR, read as "this name is not detached" and nothing more. See the From f47c2e0e3bfc5a81db5deb1e14f2622f33fbfeb1 Mon Sep 17 00:00:00 2001 From: Ettore Di Giacinto Date: Thu, 20 Aug 2026 05:20:26 +0000 Subject: [PATCH 03/11] docs(LTX25-RESIDENCY-W0): what the third review found, what closed it, and every leaf that is still unanchored MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The row's spec gains a `### What a THIRD fresh review found` section with the two mutations that broke the second repair (M12, the anchor that moves with its leaf; M11, the writer with no anchor at all), the two record findings beside them, and the seven-row mutation table re-run against this head. The second review's counts are kept and are now marked as that head's. `### Owed out of W0` used to name seven sub-millisecond leaves. It now names EVERY leaf that is not one of the four carrying phases, with the stage that acts on it: `load.dit` is order-checked and unanchored and is what W2 and W3 both act on; `conditioning.tower` and `conditioning.connector` are the 39-100% bound #1269 is about and W4 acts on them; the rest are bookkeeping on a two-block fixture, which is a claim about the fixture and not about 21B. `docs/USAGE.md` gains the two new nested names and says that the anchor check is a COUNT as well as a containment — the only check in the table that is not a ratio against the leaf, and therefore the only one an anchor that moves with its leaf cannot satisfy. The committed artifact's `_source` records that it predates both new anchors, so a reader who diffs a fresh table against it is not left to guess. The fixture's `_caveat` said "Across five such runs" and listed six walls. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code] --- .agents/specs/ltx25-device-residency.md | 167 ++++++++++++++++-- .../demo/ltx25_phase_log_fixture_cpu.json | 2 +- docs/USAGE.md | 36 ++-- 3 files changed, 174 insertions(+), 31 deletions(-) diff --git a/.agents/specs/ltx25-device-residency.md b/.agents/specs/ltx25-device-residency.md index 7c9d86b81..f6f0aa663 100644 --- a/.agents/specs/ltx25-device-residency.md +++ b/.agents/specs/ltx25-device-residency.md @@ -1000,14 +1000,15 @@ second name is inside the work". M7 fails both — seven of its eight `phase.finish` opens in the middle of the denoise window while the steps keep running around it. -**Why the sub-scope for `decode.video` sits in the driver and not in the VAE.** +**Why the sub-scope for `decode.video` sat in the driver and not in the VAE.** The per-chunk decode itself is `AccumulateTemporalGroup` in `src/vllm/model_executor/models/ltx2_video_vae_tiled.cpp`, one directory outside -this stage's authority. The driver-side anchor is weaker and it is not nothing: -its END is the production callback firing, so a `decode.video` leaf that closes -before its chunk arrives, or that is re-labelled after one, stops containing the -chunk it produced. M10 below is that case. A VAE-side sub-scope naming the tile -accumulation is **owed** and is listed under `## Owed`. +this stage's authority at the time. The driver-side anchor is weaker and it is +not nothing: its END is the production callback firing, so a `decode.video` leaf +that closes before its chunk arrives, or that is re-labelled after one, stops +containing the chunk it produced. M10 below is that case. **The VAE-side +sub-scope is no longer owed — the third review's repair placed it**, with +authority extended to that one file for the reason the next section gives. | # | Mutation | `git diff --stat` | compile | containment case | SUMS case | |---|---|---|---|---|---| @@ -1020,10 +1021,12 @@ The SUMS column is not a defect. It is the measurement: **a sum cannot see a transfer**, in three independent mutations, which is exactly why the containment case exists and why it is the one that must be run. -On the unmutated tree the containment case reports **exit 0, 1/1 cases, 124/124 -assertions**, with `denoise.step` covering **99.667%** of `denoise` over 8 -evaluations, `decode.video.chunk` **99.439%** of `decode.video` over 2 leaf -records, and mel+vocoder **99.995%** of `decode.audio`. +On the unmutated tree AT THAT HEAD the containment case reported **exit 0, 1/1 +cases, 124/124 assertions**, with `denoise.step` covering **99.667%** of +`denoise` over 8 evaluations, `decode.video.chunk` **99.439%** of `decode.video` +over 2 leaf records, and mel+vocoder **99.995%** of `decode.audio`. Every count +in this section is that head's; the section below re-runs all of them against the +third review's repair and its own numbers supersede these. **Two more findings from the same review, repaired here.** @@ -1051,6 +1054,125 @@ to say so. That is a live number in a place nobody looks — the failure this ro exists to stop, one directory over. The notice is qualitative now and the artifact's `_caveat` keeps the numbers beside the run they came from. +### What a THIRD fresh review found, and what closed it + +The second repair gave every carrying phase an anchor and asserted four things +about it: containment, coverage, exclusivity and non-overlap. **All four are +ratios taken against that anchor.** A third fresh reviewer read that and asked +the one question the previous two had not: what happens when the anchor moves +WITH the leaf. Two mutations, both compiling clean, both exit 0 on both gates. + +**M12 — the circularity, on the same phase M7 was about.** Emit `denoise.step` +for `step_index == 0` only, close `denoise` after step 0, and open +`phase.finish` over steps 1-7. The single surviving step scope sits inside the +shortened `denoise` leaf, so containment holds; it covers nearly all of it, so +coverage holds; nothing overlaps and nothing nests. The containment case reported +**exit 0, 1/1 cases, 106/106 assertions** and the SUMS case **exit 0, 350/350**, +over a table putting `phase.finish` at 28.73% of the named leaves against a +`denoise` of 6.11% on a run whose honest denoise was 39.3%. **82% of the largest +phase re-labelled onto its neighbour — M7's defect, on M7's phase, through the +gate M7 produced.** + +**M11 — the writer, which had no anchor at all.** Leave the `write(2)`s inside +`decode.video` and emit `artifacts.frames` covering nothing. The leaves stay +disjoint, nothing nests, the sum does not move, exclusivity is blind because +`artifacts.frames` is a DECLARED PARTNER of `decode.video`, and the containment +case reported **exit 0, 124/124** with `artifacts.frames` at **4 microseconds +while nine PPM files were written** (honest: 0.000186 s). The whole writer is +charged to `decode.video`, which is the leaf W5's lever is measured from. + +**F11 — the case name claimed more than it checked, and the load had nothing.** +The case was titled "each named phase CONTAINS the work it is named after" and +checked three of about fourteen. Mutation M13 swaps the `load.dit` and +`load.prompt_embeds` scope NAMES — two lines — and every gate in the file stayed +green while 96% of the load's seconds moved onto the wrong name. On the shipped +21B that name holds roughly seven and a half minutes of DiT staging, and it is +the phase W2 and W3 both act on. + +**F12 — a recorded justification pointed the wrong way.** The `denoise` coverage +margin was argued as "a percent of a 40 ms leaf", which reads as though the +uncovered part scales with the leaf. It does not: the 16 boundary samples cost +the same wall whether the denoise is 20 ms or 20 minutes, so the RATIO degrades +exactly as the hardware gets faster. The reviewer measured 99.228% on a quiet box +at a 20 ms leaf — an uncovered 0.154 ms — which puts the crossing point at a +denoise leaf of about 3.1 ms. That is a FALSE-RED risk, never a false pass. + +**F13 — the artifact's caveat said "five such runs" and listed six walls.** + +**What closed M12 and M11 needs no clock, and neither one is a ratio.** + +*The RECORD COUNT.* Each anchor must be emitted once per unit of work the RENDER +counted, and the counts come from `Ltx2ConditioningTrace`: `dit_evaluations`, +incremented inside the shared `Evaluate` lambda every sampler arm reaches, and +`video_decode_chunks`, added by this repair and incremented in the driver's own +streaming sink beside `rendered_frames`. Neither is derived from the phase table, +so no placement of a phase scope can move either, and the gate re-derives +`video_decode_chunks` from `Ltx2GroupTilesByTemporalSlice` rather than trusting +it. Under M12 the count reads `1 == 8`. + +*NOTHING BUT AN ANCHOR IS NESTED.* A leaf that grows over a NEIGHBOUR does not +overlap it and does not break the sum: `PhaseLog::Open` marks the neighbour +`nested`, which removes it from `sum_leaf_seconds` and from the timeline the +exclusivity check walks. The gate now names the six anchors and refuses any other +nested record. + +*And the WRITER got an anchor of its own.* `artifacts.frames.ppm` wraps the +`WriteFileBytes` loop, so it moves with the WRITES rather than with the scope +beside it, and `artifacts.frames` joins the carrying phases with its record count +taken from `video_decode_chunks`. Its coverage threshold is 0.50 and loose on +purpose: the leaf is 0.2 ms on this fixture and carries its own boundaries, so +what binds is the containment and the count. Measured 97.97% honest. + +*And `decode.video` got the VAE-side sub-scope this spec recorded as owed.* +`decode.video.vae` opens inside `Ltx2ConvVideoDecodeTiled` around +`AccumulateTemporalGroup`, one record per temporal group, so that leaf finally +has a sub-scope whose ends are both production events rather than statements +adjacent to its own `Open`. Authority for that one file was extended by the +operator for exactly this. + +*And the load got the only check available inside this stage's authority: its +ORDER.* An anchor for `load.dit` would have to open inside +`Ltx2LoadDitFromSafetensors` / `Ltx2StreamDitToDevice` in `ltx2_loader.cpp`, +which this stage does not reach, and the driver has one statement between that +scope's open and its close — so a sub-scope placed there would be the +adjacent-statement anchor M11 and M12 just demonstrated the circularity of. What +is asserted instead is that the load leaves appear in the order the driver runs +them, which a swapped pair of names breaks. **The case title now says what it +checks**, and `### Owed out of W0` below names every leaf whose placement is +still unproven. + +| # | Mutation | `git diff --stat` | compile | containment case | SUMS case | +|---|---|---|---|---|---| +| **M12** | emit `denoise.step` for step 0 only, close `denoise` there, open `phase.finish` over steps 1-7 — the **circularity** mutation | 1 file, +9/-1 | rc 0, 0 errors | **RED**, exit 1, 1 case failed, **167 assertions, 2 failed**, first on `CHECK( 1 == 8 )` against `dit_evaluations` | exit 0, 374/374 | +| **M11** | the writes stay inside `decode.video` and `artifacts.frames` is emitted covering nothing — the **empty-name** mutation | 1 file, +3/-4 | rc 0, 0 errors | **RED**, exit 1, 1 case failed, **194 assertions, 1 failed**, on `artifacts.frames.ppm` enclosed by no `artifacts.frames` leaf, with the leaf reporting 4.008e-06 s | exit 0, 446/446 | +| M11b | the literal reading of the same instruction: both `Close`s below the write, with `artifacts.frames` still around the loop | 1 file, +2/-2 | rc 0, 0 errors | **RED**, exit 1, 1 case failed, **126 assertions, 1 failed**, on the writer being emitted `nested` | exit 0, 446/446 | +| **M13** | swap the `load.dit` and `load.prompt_embeds` scope names | 1 file, +2/-2 | rc 0, 0 errors | **RED**, exit 1, 1 case failed, **194 assertions, 2 failed**, on the load order | exit 0, 446/446 | +| M7 | re-anchored: close `denoise` after the first sampler step and open `phase.finish` there | 1 file, +5/-0 | rc 0, 0 errors | **RED**, exit 1, 1 case failed, **195 assertions, 9 failed** | exit 0, 458/458 | +| M4 | re-anchored: leave `decode.video` open across the audio decode and give `decode.audio` a name with no work | 1 file, +3/-3 | rc 0, 0 errors | **RED**, exit 1, 1 case failed, **92 assertions, 1 failed** (a `REQUIRE_FALSE` aborts the case early) | exit 0, 446/446 | +| M10 | re-anchored: after a chunk, re-label the decode as the writer | 1 file, +1/-1 | rc 0, 0 errors | **RED**, exit 1, 1 case failed, **195 assertions, 1 failed** | exit 0, 446/446 | + +On the unmutated tree at the same head the containment case reports **exit 0, +1/1 cases, 194/194 assertions** and the SUMS case **exit 0, 1/1, 446/446**, with +`denoise.step` covering 97.81% of `denoise` over 8 evaluations, +`decode.video.chunk` 98.78% of `decode.video`, mel+vocoder 99.99% of +`decode.audio`, and `artifacts.frames.ppm` 97.97% of `artifacts.frames`. + +**The SUMS column is still not a defect, and it is now the third independent +demonstration of the same thing:** a sum cannot see a transfer, and it cannot see +an anchor that moved with its leaf either. + +**The ABI number moved, and that is the one thing in this stage a merge tool +could not have resolved.** `main` took v22 for `vllm_model_params.mmproj_path` +(#821, landed as #1436) while this branch was open, and this branch had written +v22 for `vllm_video_last_phase_log`. Two features under one version is not a +textual conflict: the version is the only question a caller can ask a loaded +library, and one that answers yes for a feature the build does not carry is worse +than none. The phase table is **v23**, and the `>= 22` floor in +`tests/capi/test_capi.cpp`, the block comments in `include/vllm.h`, the +`docs/FEATURES.md` surface row and the `docs/USAGE.md` version line and table +moved with it. The table also gained the v22 row for `mmproj_path` it was +missing. + ### No claim file, and the checker says why A `CLAIM-LTX25-RESIDENCY-W0.md` was written and then removed. @@ -1082,13 +1204,24 @@ which is where this spec's `### Decisions taken here` already said they would be the rank of its two dominant phases. Until W1 takes a lease on an idle box, no DURATION in any phase table this row produced may be compared with any other duration, including its own from the run before. -* **The sub-millisecond leaves are named, not measured.** `generate.setup`, - `generate.geometry`, `generate.guiders`, `generate.image_cond`, - `generate.audio_input`, `generate.retake` and `phase.finish` are each under - 0.001% of the leaf sum on the fixture; F2's containment and floor cover the - three phases that carry the render and say nothing about these. That is a fact - about the driver — they really are bookkeeping — but it means their PLACEMENT - is unproven, and a shipped-checkpoint render is where it would show. +* **EVERY LEAF THAT IS NOT ONE OF THE FOUR CARRYING PHASES IS NAMED AND NOT + ANCHORED**, and this entry used to list only seven of them. The gate anchors + `denoise`, `decode.video`, `decode.audio` and `artifacts.frames` — one nested + sub-scope per unit of work the render counted, contained in the leaf, covering + it, exclusive of other leaves. **Nothing else in the table has an anchor.** The + full list, so that no later stage inherits an unproven placement by silence: + + | Leaf | State | Who acts on it | + |---|---|---| + | `load.dit` | **order only.** Its placement is unproven, and mutation M13 shows what that costs: swapping its name with `load.prompt_embeds` moved 96% of the load's seconds and reddened nothing until the order check landed. An anchor needs a scope inside `Ltx2LoadDitFromSafetensors` / `Ltx2StreamDitToDevice` (`ltx2_loader.cpp`), outside this stage's authority | **W2 and W3, both** | + | `load.video_vae`, `load.audio_vae`, `load.upsampler`, `load.text_encoder`, `load.prompt_embeds` | order only, same argument | W2 | + | `conditioning.tower`, `conditioning.connector` | unanchored. These are the 39-100% bound [#1269](https://github.com/mudler/vllm.cpp/issues/1269) is about, so W4 acts on two numbers whose placement no gate holds | **W4** | + | `generate.setup`, `generate.geometry`, `generate.guiders`, `generate.image_cond`, `generate.audio_input`, `generate.retake`, `generate.trim`, `phase.finish`, `phase.prepare` | unanchored, and each under 0.001% of the leaf sum on the fixture. That is a fact about the DRIVER — they really are bookkeeping — and it is a fact about a two-block fixture, not about 21B | nobody, today | + | `phase.upsample_latent` | unanchored, and gated only on the reduced two-phase fixture (see below) | W2c | + | `artifacts.audio` | unanchored; the WAV write, one call | nobody, today | + + The `load` and `generate` records are SPANS and are never summed, so they are + not in this list; `sum_rule` in every emitted table says which records add up. * **F10 IS NOW GATED, and the sentence that said it could not be was wrong.** This entry used to read "F10 is an absence — a thread that keeps running — which no assertion in this suite is positioned to observe", and asserted that a diff --git a/benchmarks/demo/ltx25_phase_log_fixture_cpu.json b/benchmarks/demo/ltx25_phase_log_fixture_cpu.json index 48f86fc0d..c7d9d91c2 100644 --- a/benchmarks/demo/ltx25_phase_log_fixture_cpu.json +++ b/benchmarks/demo/ltx25_phase_log_fixture_cpu.json @@ -1,5 +1,5 @@ { - "_source": "MEASURED 2026-08-20 on mudler-ubuntu-box (Linux 6.8.0-136-generic x86_64, 20 cores, CONTENDED -- load average 31 to 55 while this ran, other sessions compiling throughout). Produced by `build/examples/ltx2-gen`, the shipped example, which is a client of `vllm.h` and nothing else: it calls `vllm_video_engine_load` + `vllm_video_generate` and asks `vllm_video_last_phase_log` for this file's path. Geometry `--frames 9 --width 64 --height 64 --seed 7 --max-phase 0 --device cpu` over the reduced fixture. Built from 3dc2ae98b on row/LTX25-RESIDENCY-W0 with `cmake -DCMAKE_BUILD_TYPE=Release -G Ninja`, gcc 13.3.0. Every byte below `_source`, `_caveat`, `_headline` and `_footnotes` is the render's own output. It REPLACES the artifact taken at e3f46560e, which predates the `denoise.step` and `decode.video.chunk` anchors and therefore could not show them.", + "_source": "MEASURED 2026-08-20 on mudler-ubuntu-box (Linux 6.8.0-136-generic x86_64, 20 cores, CONTENDED -- load average 31 to 55 while this ran, other sessions compiling throughout). Produced by `build/examples/ltx2-gen`, the shipped example, which is a client of `vllm.h` and nothing else: it calls `vllm_video_engine_load` + `vllm_video_generate` and asks `vllm_video_last_phase_log` for this file's path. Geometry `--frames 9 --width 64 --height 64 --seed 7 --max-phase 0 --device cpu` over the reduced fixture. Built from 3dc2ae98b on row/LTX25-RESIDENCY-W0 with `cmake -DCMAKE_BUILD_TYPE=Release -G Ninja`, gcc 13.3.0. Every byte below `_source`, `_caveat`, `_headline` and `_footnotes` is the render's own output. It REPLACES the artifact taken at e3f46560e, which predates the `denoise.step` and `decode.video.chunk` anchors and therefore could not show them. This capture in turn PREDATES the third review's two further anchors -- `decode.video.vae`, one nested record per temporal group the tiled VAE decodes, and `artifacts.frames.ppm`, one per write callback -- so a table emitted by a later build carries two record kinds this one does not. Nothing else about the shape changed, and no duration here may be compared with one from any other run in either direction.", "_caveat": "NOT A BENCHMARK, AND NOT A PROFILE OF ANY SHIPPED CHECKPOINT. The weights are the REDUCED two-block fixture `tests/vllm/multimodal/ltx2_video_fixture.h` writes, not Lightricks' 21.00B DiT, so no duration here describes a real render. The host was contended, and the sharpest evidence for that is in this file's own history: the run taken ONE MINUTE before this one, from the same binary at the same geometry, measured 0.147 s of wall against this run's 4.463 s. Across six such runs the wall has read 0.147 s, 0.158 s, 1.676 s, 4.463 s, 6.138 s and 12.030 s, and the RANK of the two largest phases has reversed between them. Nothing in this file may be quoted as a speed, a ratio between phases, or an ordering. What it supports is the SHAPE of the table -- which phases exist, that they do not overlap, that each carrying phase CONTAINS its own sub-scopes, and that they add up. The `notice` field says the same thing qualitatively and travels with every phase log the library writes.", "_headline": "99.94% of an LTX-2.5 render's wall now has a phase name on it, over 33 entries, and each of the three phases that carry the render encloses the sub-scopes that name its work.", "_footnotes": [ diff --git a/docs/USAGE.md b/docs/USAGE.md index 46ed44e46..58c0e340e 100644 --- a/docs/USAGE.md +++ b/docs/USAGE.md @@ -1084,19 +1084,29 @@ count. These fields say how complete it is, and how far it carries: Some phases are **decomposed rather than partitioned**. `denoise` carries one `denoise.step` per denoiser evaluation, `decode.video` carries -`decode.video.chunk` per streamed chunk, `decode.audio` carries -`decode.audio.mel` and `decode.audio.vocoder`, and a two-stage recipe's -`phase.prepare` carries `phase.upsample_latent`. Those records are marked -`nested`, are printed for the reader, and are **excluded from -`sum_leaf_seconds`** — they are inside a leaf that is already counted, so adding -them would make `unaccounted_seconds` the residue of double counting instead of -time nobody named. - -A nested record is also what makes a phase NAME checkable. A leaf that claims to -cover the denoise must enclose its own `denoise.step` records; one that stops -short of the loop, or that hands the back half of it to a neighbouring name, no -longer does. The three phases that carry a render each carry such an anchor for -that reason. +`decode.video.chunk` per streamed chunk and `decode.video.vae` per temporal +group the tiled VAE decodes, `decode.audio` carries `decode.audio.mel` and +`decode.audio.vocoder`, `artifacts.frames` carries `artifacts.frames.ppm` per +write callback, and a two-stage recipe's `phase.prepare` carries +`phase.upsample_latent`. Those records are marked `nested`, are printed for the +reader, and are **excluded from `sum_leaf_seconds`** — they are inside a leaf +that is already counted, so adding them would make `unaccounted_seconds` the +residue of double counting instead of time nobody named. + +A nested record is also what makes a phase NAME checkable, and it is checkable in +two ways rather than one. A leaf that claims to cover the denoise must ENCLOSE +its own `denoise.step` records; one that stops short of the loop, or that hands +the back half of it to a neighbouring name, no longer does. And the anchor must +appear once per unit of work the RENDER counted — one `denoise.step` per +denoiser evaluation, one `decode.video.chunk` per streamed chunk plus the reopen +after the last one — which is the only check here that is not a ratio against the +leaf, and therefore the only one an anchor that moves WITH its leaf cannot +satisfy. The four phases that carry a render each carry such an anchor. + +A record that is `nested` and is NOT one of those anchors means a leaf was left +open across a phase it does not name: the neighbour turns `nested`, leaves +`sum_leaf_seconds`, and overlaps nothing, so a table can lose a whole phase to +its neighbour without any two intervals crossing. **Do not read a duration here as a measurement of this machine.** Every number is wall clock under whatever else the box was doing, which the file does not From 165db635cbbb2e437d3b9e6deb6f7b8943f8f41d Mon Sep 17 00:00:00 2001 From: Ettore Di Giacinto Date: Thu, 20 Aug 2026 05:27:45 +0000 Subject: [PATCH 04/11] record(LTX25-RESIDENCY-W0): the third review's findings get an issue, and the index gets its row `#1440` tracks what the third fresh review of #1408 found: four of the containment case's five assertions are ratios against the sub-scope anchor, so an anchor that moves with its leaf satisfies all of them. Filed and fixed in the same flow, per AGENTS.md, with the row appended to the index and the issue named from the spec's `## Owed` table and its new review section. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code] --- .agents/issue-index.md | 1 + .agents/specs/ltx25-device-residency.md | 4 ++++ 2 files changed, 5 insertions(+) diff --git a/.agents/issue-index.md b/.agents/issue-index.md index 71e04668c..85e7ff298 100644 --- a/.agents/issue-index.md +++ b/.agents/issue-index.md @@ -472,3 +472,4 @@ rather than merged. `scripts/check-agent-record.py` gates both. | [#1433](https://github.com/mudler/vllm.cpp/issues/1433) | `MODEL-DIFFUSION-LTX25` | `Lightricks/LTX-2` is the repository the WHOLE LTX-2.5 lane mirrors — ten `.agents/specs/ltx25-*.md` files and `.agents/porting-inventory.md` carry the revision string `fd4ded7f`, and every `file:line` upstream anchor in that lane resolves there — and it has NO registry row in `AGENTS.md`, NO `.agents/oracles/.md` file, and NO recorded pin. Measured at `2da236832`: `scripts/check-oracle-pins.py` reports `oracle-pins ok (9 oracles pinned)` — `diffusers`, `llama-cpp`, `llama-cpp-unsloth`, `sglang`, `sglang-omni`, `transformers`, `tt-forge`, `vllm`, `vllm-omni` — and it reads `.agents/oracles/*.md` with a GLOB, so an oracle with no file is INVISIBLE to it. The green line is what an unpinned upstream looks like from inside the gate, which is the failure `AGENTS.md` "Pin every oracle" names: "An unpinned upstream is a moving target, not an oracle. Its measurements are not reproducible." `vllm-omni` does NOT already cover it: `.agents/oracles/vllm-omni.md:5` names LTX-2.5 in scope and is the right home for the MIRRORING relationship (`vllm-project/vllm-omni` is the structure this project mirrors), but it is not the repository the specs READ — that is a THIRD repository — and `vllm-omni` is itself `pin = UNPINNED` under [#633](https://github.com/mudler/vllm.cpp/issues/633), so the lane's actual reference source is unpinned twice over, once knowingly and once with no record at all. PRE-EXISTING and NOT caused by [#1428](https://github.com/mudler/vllm.cpp/pull/1428): the `fd4ded7f` anchoring dates to the original port `cefacd2d0` (2026-08-13, #435 / #641), first carried by `f193eaf75`, `742e38acf` and `0541cbeaa`. NOT FIXED IN FLOW because adding a row to the `AGENTS.md` oracle table is a POLICY edit, which `AGENTS.md` "Changing the rules or a checker" routes to its own row, spec and fresh review. Found while repairing the fresh review of #1428. Owned by row `MODEL-DIFFUSION-LTX25`; spec [`ltx-2-5.md`](specs/ltx-2-5.md) | gap | | [#1435](https://github.com/mudler/vllm.cpp/issues/1435) | `VT-MATMUL-FP8-BLOCK-CUDA` | **A CUDA build without CUTLASS headers segfaults on the block-wise FP8 path instead of refusing by name, and configure reports the feature ENABLED anyway.** Measured 2026-08-20 on `dgx:gpu0` (GB10, cc 12.1) in an `rc` lease at `63d87805c`. `src/vt/cuda/cuda_matmul_fp8_block_cutlass.cu` is the only TU that registers `kMatmulFp8BlockScaled` for `kCUDA` and it reaches `target_sources(vllm PRIVATE ... ${_FP8_CUTLASS_SOURCES})` only through TWO nested conditions in `CMakeLists.txt` -- the outer `if(VLLM_CPP_CUTLASS)` (one occurrence in the file) and, inside it, the `if(VT_CUTLASS_FP8_ARCHS)` that SETS `_FP8_CUTLASS_SOURCES` -- so with no CUTLASS headers the op is unregistered, a device tensor reaches the portable HOST kernel, and the process SIGSEGVs while the reference tier prints `correct but slow`. Anchored by SYMBOL rather than by line on purpose, and the reason is measured on this very row: the guard sat at `CMakeLists.txt:1695@63d87805c` -- the only line number in this row, sha-qualified so it stays resolvable -- and by the time this row was written that number was already stale, then rotted AGAIN by four lines when `origin/main` was merged into the row's own branch mid-review. Two rots inside one pull request. A bare head-relative number here would be permanent, because this index is append-only and no checker scans it, and it would resolve to plausible unrelated code rather than to nothing: at `63d87805c` that line was the guard, and two trees later it was an NCCL include directory. Resolve the anchors above by grepping the symbols; do not trust any number quoted for them. Reachable on a default build: `VLLM_CPP_CUTLASS_FETCH` defaults OFF (the single `option(VLLM_CPP_CUTLASS_FETCH ...)` line, `OFF` at its default argument) and CUTLASS is not a submodule. This contradicts the design note in `cuda_matmul_fp8_block_cutlass.cu`, which states an unsupported build `keeps refusing by name -- which is the honest answer and not the #960/#844 fall-through`: the refusal IS real on the model path (`RefuseUnrunnableQwen3_5DenseFp8Block`, plus the two `VT_CHECK`s) but a caller entering through the op registry, which is what the ported upstream test does, bypasses all three. Second, smaller defect in the same run: `vt_cuda_report_feature` printed `CUDA feature cutlass-fp8: ENABLED for [121a]` for a build whose `build.ninja` contains ZERO references to either FP8 CUTLASS TU, because the report runs on the arch intersection alone, before CUTLASS-header detection. `cmake/CudaArchFeatures.cmake` opens by naming this exact class as what the feature table exists to prevent. NOT fixed in flow: both are code changes on a CUDA path needing their own row, spec and hardware re-gate. Owned by row `VT-MATMUL-FP8-BLOCK-CUDA`, under `## Owed` in [`vt-matmul-fp8-block-cuda.md`](specs/vt-matmul-fp8-block-cuda.md) | bug | | [#1437](https://github.com/mudler/vllm.cpp/issues/1437) | `VT-MATMUL-FP8-BLOCK-CUDA` | **The mainloop-scaled CUTLASS block-wise FP8 GEMM throws `cutlass Invalid status` on upstream's own ported case, on the arch it targets.** FIRST on-hardware execution of #1189 M5 (`489a9a4c0`), measured 2026-08-20 on `dgx:gpu0` (GB10, driver 580.173.02, compute capability 12.1) in an `rc` lease at `63d87805c`, CUDA 13.0 Release, `-DVLLM_CPP_CUDA_ARCHITECTURES=121a -DVLLM_CPP_CUTLASS_FETCH=ON`. G2 (upstream's `test_w8a8_block_fp8_cutlass_matmul` ported whole: M=32, N=576, K=7168, block [128,128], bf16 out) and G7 both THROW from `gemm_op.can_implement(args)`, so CUTLASS rejects the configuration before any launch -- not a numerical disagreement and not a launch failure. G6, G8 and G9 PASS. That this was an EXECUTION and not a skip rests on the throw TEXT, not on a count: `vt cuda: matmul_fp8_block_scaled: cutlass Invalid status` is assembled by the `VT_CUTLASS_CHECK` macro in `src/vt/cuda/cuda_matmul_fp8_block_cutlass.cu`, and the message prefix `matmul_fp8_block_scaled: cutlass ` occurs in NO other translation unit (the per-tensor sibling `cuda_matmul_fp8_cutlass.cu` defines a same-named macro but writes a different message), in a TU compiled only when CUTLASS is found, so no host fallback can produce that string; the run also logged ZERO `[vt reference-tier]` lines, and artifact provenance was confirmed by `cuobjdump --list-elf` reporting `cuda_matmul_fp8_block_cutlass.cu.1.sm_121a.cubin`. Suite: 5 cases, 3 passed, 2 failed, **34 assertions, 0 failed** -- recorded, but attributed rather than leaned on, because **27** of the 34 are G6, which has no `HasCuda()` guard and prints identically on a GPU-less box (8 grid entries x 3 `CHECK` + 3 tile-config `CHECK`s); the device-only remainder is G8's 2 and G9's 5, and a pure skip of this file prints 27. NO shape has had its output compared with the CPU reference: G2 and G7 are the only two cases that make that comparison and BOTH threw before their first assertion -- G7 aborted at `Grid()[0]`, the same M=32 N=576 K=7168 case, so its other seven shapes, including the second ragged-N entry M=8 N=576 K=1024, were never attempted -- and G8 compares a device f32 out against a device bf16 out, which is the kernel against ITSELF. Hypothesis recorded in the issue and explicitly NOT isolated: N=576 is 4*128+64, a ragged final scale block, which the test header records as upstream's reason for choosing it; G8 passing makes the failure shape-dependent rather than universal. This moves the row from `unmeasured` to `measured and failing`, a worse position than the spec previously described. NO speed claim: the lease took no clock control and recorded no contention. Owned by row `VT-MATMUL-FP8-BLOCK-CUDA`, under `## Owed` in [`vt-matmul-fp8-block-cuda.md`](specs/vt-matmul-fp8-block-cuda.md) | measurement | +| [#1440](https://github.com/mudler/vllm.cpp/issues/1440) | `LTX25-DEVICE-RESIDENCY` | **W0's phase-table gate cannot see an anchor that moves WITH its leaf.** Found by the third fresh review of [#1408](https://github.com/mudler/vllm.cpp/pull/1408) against `89261c955`. The containment case asserts containment, coverage, exclusivity and non-overlap for each carrying phase, and all four are RATIOS against the sub-scope anchor, so a defect that moves the leaf and its anchor together satisfies every one of them. M12 emits `denoise.step` for step 0 only, closes `denoise` there and opens `phase.finish` over steps 1-7: **exit 0, 106/106 and 350/350**, over a table putting `phase.finish` at 28.73% against a `denoise` of 6.11% on a run whose honest denoise was 39.3% -- 82% of the largest phase re-labelled, which is M7's defect on M7's phase through the gate M7 produced. M11 leaves the `write(2)`s inside `decode.video` and emits `artifacts.frames` covering nothing: **exit 0, 124/124 and 422/422**, coverage 99.68%, with `artifacts.frames` reporting 4 microseconds for nine PPM files and the whole writer charged to the leaf W5's lever is measured from. Three smaller findings beside them: the case was titled 'each named phase CONTAINS the work it is named after' and checked three of about fourteen, with `load.dit` -- the phase W2 and W3 both act on, ~7.5 minutes of DiT staging on the shipped 21B -- unanchored and its name swappable with `load.prompt_embeds` for 96% of the load's seconds with every gate green; `### Owed out of W0` named seven sub-millisecond leaves and omitted `load.*`, `conditioning.tower` and `conditioning.connector`, the last two being #1269's 39-100% bound that W4 acts on; and the `denoise` coverage margin was justified as 'a percent of a 40 ms leaf' when the per-boundary cost is FIXED while the leaf shrinks, measured 99.228% at a 20 ms leaf against a 0.95 threshold, which is a false-RED risk on faster hardware rather than a false pass. NOT a correctness defect in the instrument, which all three reviews found reachable on the shipped default. Fixed in flow. Spec [`ltx25-device-residency.md`](specs/ltx25-device-residency.md) `### What a THIRD fresh review found` | bug | diff --git a/.agents/specs/ltx25-device-residency.md b/.agents/specs/ltx25-device-residency.md index f6f0aa663..9341ab893 100644 --- a/.agents/specs/ltx25-device-residency.md +++ b/.agents/specs/ltx25-device-residency.md @@ -635,6 +635,7 @@ its phase table lands, and W5 owes one when its wall is accepted. | Issue | Stage | State | |---|---|---| | [#1264](https://github.com/mudler/vllm.cpp/issues/1264) | this row: the staged campaign spec | closed by this row landing | +| [#1440](https://github.com/mudler/vllm.cpp/issues/1440) | W0 (gate) | **closed by the third review's repair.** Four of the containment case's five assertions were ratios against the sub-scope anchor, so an anchor that moved with its leaf satisfied all of them; the repair adds a record COUNT taken from the render's own counters and a rule that nothing but an anchor may be emitted `nested`. See `### What a THIRD fresh review found` | | [#1010](https://github.com/mudler/vllm.cpp/issues/1010) | W0 | **closed, for a run that FINISHES.** The render writes a phase table on the shipped default and the ABI names it through `vllm_video_last_phase_log` (v23). Read the row below it before quoting that as "the render is instrumented" | | **a table on a run that does NOT finish** | [#1413](https://github.com/mudler/vllm.cpp/issues/1413), stage W0-live | **owed, and W0 does not narrow it.** `WritePhaseLog` has exactly two call sites (`ltx2_video.cpp:2259` audio-only, `:4677` video) and both sit immediately before a successful `return`, three lines after `im.trace.completed = true`. Nineteen `VT_CHECK` sites throw out of `Ltx2VideoEngine::Generate` above them and that body contains no `try` and no `catch`; `vllm_video_generate`'s own two catches set an error and return, and `engine->last_phase_log` is assigned on the success path only. **So a render that is killed, aborted by a lease governor, refused by a guard, or still running leaves no `phase-log.json` at all** — not a truncated one, not an empty one, nothing. The mutation is the demonstration: deleting the video call site removes the file entirely and the W0 gate goes red on `REQUIRE(probe.good())`. That matters here more than anywhere, because the runs this campaign has are the ones that died: [#1375](https://github.com/mudler/vllm.cpp/issues/1375) is `child exit=-15` at 0 frames, and [`ltx25-decode-speed.md`](ltx25-decode-speed.md)'s two rungs are `EXIT=137` and `EXIT=1` at 0 frames. A reader who takes "the render writes a phase table" at face value will expect a 2.5 h render that is killed at 2.4 h to leave a table naming where it was. It leaves none. #1413 closes the LIVE half — a line per phase boundary and per DiT forward, so a killed run is legible from its stderr; a signal handler that flushed a PARTIAL table on abort is a separate change with its own re-entrancy argument and is not owed by either | | the phase table's DEVICE column | W1, and it needs [#1126](https://github.com/mudler/vllm.cpp/issues/1126) first | **owed, and a LEASE WILL NOT CLOSE IT.** The column is defined as the driver's live in-use bytes, read per phase through `vt::Backend::DeviceMemoryInfo`. It reports the `-1` no-probe sentinel in W0's artifact, and there are TWO reasons stacked, only one of which is a scheduling problem. **(1)** The render W0 could take was the CPU arm, where the sentinel is correct — `dgx:gpu0` was busy with two queued jobs and `orin:gpu0`, the one free device, holds no LTX-2.5 checkpoints. **(2)** The one that matters: **`CudaBackend` does not override `DeviceMemoryInfo` at all**, which is [#1126](https://github.com/mudler/vllm.cpp/issues/1126), and `grep -rn 'DeviceMemoryInfo' src/vt include/vt` returns exactly the base declaration at `include/vt/backend.h:94` and one override, `src/vt/rocm/rocm_backend.hip:358`. So a CUDA render on `dgx:gpu0` would print `-1` in every row of this column too, and it would print it for a reason no lease can fix. **What the column cannot report today, stated as three things:** how many device bytes the DiT staging leaves resident; whether the denoise grows device residency across steps; and whether the ~59 GiB #1014 asks about is device-class at all. **Why W0 did not just wire it:** `include/vllm/platforms/interface.h:68-72` records that CUDA's absence from that seam is load-bearing — `Gemma4MoE`'s device-expert LRU is the seam's only consumer and is DEAD on CUDA. Narrowed, because the broad form of this sentence is wrong: the bf16 arm `EnsureGemma4Fp8ExpertOnDevice` (`gemma4_moe.cpp:548`) is dead for a SECOND, independent reason — it refuses at `:571` on `vt::HasMatmulBTAlphaBeta`, implemented only by ROCm ([#1205](https://github.com/mudler/vllm.cpp/issues/1205)) — so the override alone would not wake it. The arm the override WOULD wake is the FP8-native one, `EnsureGemma4Fp8NativeOnDevice` (`:611-628`), which reaches `MakeRoom` behind no device gate but the probe, with a 2048 MiB fill-only budget by default (`:416-432`). One named arm, not every CUDA model — and still a behaviour change with its own measurement. That is #1126's change to make, not an instrument's. **What W1 does instead, until #1126 lands:** sample `nvidia-smi --query-compute-apps=used_memory` per phase beside the table — the fallback this spec's W6 row already names, and the one instrument GB10 answers, since `--query-gpu=memory.used` returns `[N/A]` there. On GB10 the peak HOST column is not a poor substitute either: the pool is unified, so host resident bytes and device bytes are the same 119 GiB arena, and that column does report | @@ -1056,6 +1057,9 @@ artifact's `_caveat` keeps the numbers beside the run they came from. ### What a THIRD fresh review found, and what closed it +Tracked by [#1440](https://github.com/mudler/vllm.cpp/issues/1440), filed and +fixed in the same flow. + The second repair gave every carrying phase an anchor and asserted four things about it: containment, coverage, exclusivity and non-overlap. **All four are ratios taken against that anchor.** A third fresh reviewer read that and asked From 821ef54b61666e93816fde7918b51881fab99461 Mon Sep 17 00:00:00 2001 From: Ettore Di Giacinto Date: Thu, 20 Aug 2026 06:19:23 +0000 Subject: [PATCH 05/11] fix(LTX25-RESIDENCY-W0): the two anchors a fourth review could still swap and empty, and the counts get an N greater than 1 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A fourth fresh review broke two of this gate's own stated guarantees with two-line mutations, and both holes are in the anchored set the PR declares. SIBLING NAMES WERE SWAPPABLE. Nothing in the case tied a sub-scope NAME to a POSITION: the counts are per name, containment holds for either, `subs` is sorted by start before the overlap check, and `CheckOnlyAnchorsAreNested` is a set-membership test. Swapping the `decode.audio.mel` and `decode.audio.vocoder` string literals in the driver moved 96.8% of `decode.audio`'s decomposed seconds onto the wrong name with every assertion green, on a leaf holding 47.8% of that render's named leaf seconds and that #1010 asks for by name. Assertion (1b) is the same shape assertion (6) already uses for the load: sibling sub-scopes appear in the order `parts` lists them, compared on each name's first record, so a swapped PAIR is a red. `decode.video.vae` WAS HELD BY A CARDINALITY ALONE. Its duration was compared against nothing — `Carrying{"decode.video", ...}` lists only `decode.video.chunk` in `parts`, so the vae contributed zero to any coverage. Moving the scope off `AccumulateTemporalGroup` and onto the `buffer.Allocate` beside it left the table reporting `decode.video.vae = 0.000 s` beside a five-millisecond `decode.video` and both cases green. It now has a coverage floor against `decode.video.chunk` seconds, measured 91.1%-98.6% over six renders and floored at 0.50. AND THE COUNTS ARE DEMONSTRATED AT N > 1. Every decode-side count was 1 or 1+1, because nine frames decode in one temporal chunk, so "once per unit of work" and "once per render" were indistinguishable. The case now renders a second time at 81 frames — the smallest request this fixture chunks — and re-runs every per-render assertion against a table that holds both. That makes the new `render` filter load-bearing rather than hypothetical: `PhaseLog` resets at LOAD while `Ltx2ConditioningTrace` resets per `Generate`, so an unfiltered count is process-wide against a per-render denominator. The tiling re-derivation also reads the fixture's own decoder blocks and the size the render reported, in place of three constants copied from a header comment. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code] --- tests/vllm/multimodal/test_ltx2_video.cpp | 396 +++++++++++++++++----- 1 file changed, 313 insertions(+), 83 deletions(-) diff --git a/tests/vllm/multimodal/test_ltx2_video.cpp b/tests/vllm/multimodal/test_ltx2_video.cpp index c900afb19..d44963781 100644 --- a/tests/vllm/multimodal/test_ltx2_video.cpp +++ b/tests/vllm/multimodal/test_ltx2_video.cpp @@ -2993,15 +2993,42 @@ TEST_CASE("ltx2 video: a render through the ABI emits a phase table that SUMS to // detached", never as "this phase is honest". namespace { -// Every record of one name, in emitted (start-sorted) order. -std::vector RecordsNamed(const nlohmann::json& table, const std::string& name) { +// "any render", for a lookup that genuinely spans the whole table. +constexpr int64_t kAnyRender = -1; + +// Every record of one name IN ONE RENDER, in emitted (start-sorted) order. +// +// THE RENDER ARGUMENT IS NOT DECORATION, and a fourth fresh review named the +// hole it closes. `PhaseLog` is a PROCESS timeline that resets at LOAD +// (`ltx2_video.cpp`, `Begin()` in the load path); `Ltx2ConditioningTrace` resets +// on every `Generate` (`ltx2_video.cpp`, `im.trace = Ltx2ConditioningTrace{}`). +// So a count taken over every record of a name is process-wide while every +// denominator it is compared against is per-render, and the two agreed only +// while this case rendered exactly once. It now renders twice into one table, +// and without this filter the second render's counts would be the sum of both. +std::vector RecordsNamed(const nlohmann::json& table, const std::string& name, + int64_t render = kAnyRender) { std::vector out; for (const nlohmann::json& e : table["phases"]) { - if (e["name"].get() == name) out.push_back(e); + if (e["name"].get() != name) continue; + if (render != kAnyRender && e.value("render", static_cast(0)) != render) continue; + out.push_back(e); } return out; } +// The render index of the LAST generation in a table. The load is render 0 and a +// table written by the second `Generate` of a process holds the load's records +// and BOTH renders', so "this render" is a value the reader has to resolve +// rather than assume. +int64_t LastRender(const nlohmann::json& table) { + int64_t last = 0; + for (const nlohmann::json& e : table["phases"]) { + last = std::max(last, e.value("render", static_cast(0))); + } + return last; +} + // Leaf seconds per NAME, summed: `decode.video` alternates with // `artifacts.frames` and appears more than once per render, so a per-name total // is the only quantity that means anything. Leaves only — spans enclose leaves @@ -3021,11 +3048,13 @@ struct NamedInterval { double end = 0.0; }; -// Every SUMMED leaf as an interval, in start order. -std::vector LeafIntervals(const nlohmann::json& table) { +// Every SUMMED leaf of one render as an interval, in start order. +std::vector LeafIntervals(const nlohmann::json& table, + int64_t render = kAnyRender) { std::vector out; for (const nlohmann::json& e : table["phases"]) { if (e.value("span", false) || e.value("nested", false)) continue; + if (render != kAnyRender && e.value("render", static_cast(0)) != render) continue; NamedInterval iv; iv.name = e["name"].get(); iv.start = e["start_seconds"].get(); @@ -3049,6 +3078,9 @@ struct Carrying { // note above this case. std::vector part_counts; std::string count_source; // where those numbers came from, for the message + // WHICH RENDER. The table is a process timeline and every count above is a + // per-render quantity; see `RecordsNamed`. + int64_t render = kAnyRender; }; // The four assertions above, for one carrying phase. A free function rather than @@ -3057,7 +3089,7 @@ struct Carrying { // `denoise` came to be ungated in the first place. void CheckCarryingPhase(const nlohmann::json& table, const Carrying& c) { INFO("carrying phase = " << c.leaf); - const std::vector leaves = RecordsNamed(table, c.leaf); + const std::vector leaves = RecordsNamed(table, c.leaf, c.render); REQUIRE_MESSAGE(!leaves.empty(), "the table names no '" << c.leaf << "' leaf at all"); double leaf_seconds = 0.0; for (const nlohmann::json& r : leaves) { @@ -3084,7 +3116,8 @@ void CheckCarryingPhase(const nlohmann::json& table, const Carrying& c) { // of times than the work it claims to wrap is caught whatever the clock did. REQUIRE(c.part_counts.size() == c.parts.size()); for (size_t i = 0; i < c.parts.size(); ++i) { - const int64_t emitted = static_cast(RecordsNamed(table, c.parts[i]).size()); + const int64_t emitted = + static_cast(RecordsNamed(table, c.parts[i], c.render).size()); INFO("sub-scope = " << c.parts[i]); CHECK_MESSAGE(emitted == c.part_counts[i], "'" << c.parts[i] << "' was emitted " << emitted << " time(s) and the render " @@ -3096,9 +3129,12 @@ void CheckCarryingPhase(const nlohmann::json& table, const Carrying& c) { // (1) CONTAINMENT, and the sub-scopes' own intervals. std::vector> subs; + // ...and the FIRST start of each part name, in the order `parts` lists them, + // which is assertion (1b) below. + std::vector part_first; for (const std::string& part : c.parts) { INFO("sub-scope = " << part); - const std::vector found = RecordsNamed(table, part); + const std::vector found = RecordsNamed(table, part, c.render); REQUIRE_MESSAGE(!found.empty(), "the table names no '" << part << "' record. It is the anchor that ties the '" << c.leaf << "' name to the work beneath it, and without it this phase is " @@ -3125,8 +3161,46 @@ void CheckCarryingPhase(const nlohmann::json& table, const Carrying& c) { "case exists for"); subs.push_back({start, end}); } + double first = found.front()["start_seconds"].get(); + for (const nlohmann::json& r : found) { + first = std::min(first, r["start_seconds"].get()); + } + part_first.push_back(first); } REQUIRE(!subs.empty()); + + // (1b) THE SIBLING SUB-SCOPES RUN IN THE ORDER `parts` LISTS THEM, which is + // the order the driver runs them in. + // + // WHY IT EXISTS, and it is the same defect assertion (6) was added for one + // review earlier. NOTHING ABOVE TIES A SUB-SCOPE NAME TO A POSITION: the + // counts are per name, containment holds for either name, `subs` is sorted by + // start BEFORE the overlap check, and `CheckOnlyAnchorsAreNested` is a + // set-membership test. So swapping the two string literals at the + // `decode.audio.mel` / `decode.audio.vocoder` scopes in the driver — two + // lines — moved 96.8% of `decode.audio`'s decomposed seconds onto the wrong + // name with every assertion in this file green, on a leaf that held 47.8% of + // that render's named leaf seconds and that #1010 asks for BY NAME. W1 would + // have ranked the wrong model. + // + // WHAT IT PROVES AND WHAT IT DOES NOT. It proves that the names appear in the + // driver's own order, so a swapped PAIR is a red. It does NOT prove that + // either name covers the call it is named after; that would need an anchor + // inside `Ltx2AudioDecoderForward` / `Ltx2VocoderWithBweForward`, which is the + // same shape `load.dit` is owed and is recorded beside it in the row's spec + // under `### Owed out of W0`. Compared on the FIRST record of each name, so a + // part that legitimately repeats (`decode.video.chunk`) is unaffected, and the + // assertion is vacuous for a leaf with a single part name. + for (size_t i = 1; i < part_first.size(); ++i) { + INFO("sub-scope pair = " << c.parts[i - 1] << " then " << c.parts[i]); + CHECK_MESSAGE(part_first[i] >= part_first[i - 1] - 1e-9, + "'" << c.parts[i] << "' first runs at " << part_first[i] << " and '" + << c.parts[i - 1] << "' at " << part_first[i - 1] << ", so the table " + << "reports the sub-scopes of '" << c.leaf << "' in an order the driver " + "does not run them in. Two SIBLING scope names that were swapped move " + "their seconds onto each other and change nothing else in this file"); + } + std::sort(subs.begin(), subs.end()); // The sub-scopes decompose one leaf, so they run in sequence and never overlap // each other. Two that did would double count into the coverage below. @@ -3156,7 +3230,7 @@ void CheckCarryingPhase(const nlohmann::json& table, const Carrying& c) { // steps keep running around it. const double window_start = subs.front().first; const double window_end = subs.back().second; - for (const NamedInterval& iv : LeafIntervals(table)) { + for (const NamedInterval& iv : LeafIntervals(table, c.render)) { if (iv.name == c.leaf) continue; if (std::find(c.partners.begin(), c.partners.end(), iv.name) != c.partners.end()) continue; // One bool, because doctest refuses to decompose a `||` inside an assertion. @@ -3211,10 +3285,10 @@ void CheckOnlyAnchorsAreNested(const nlohmann::json& table, // keeps the `write(2)` off W5's lever. Asserted over the INTERVALS as well as // over the `nested` flag, so a change in what the emitter means by `nested` // cannot quietly retire it. -void CheckWriterIsBesideTheDecode(const nlohmann::json& table) { - const std::vector writes = RecordsNamed(table, "artifacts.frames"); +void CheckWriterIsBesideTheDecode(const nlohmann::json& table, int64_t render) { + const std::vector writes = RecordsNamed(table, "artifacts.frames", render); REQUIRE_MESSAGE(!writes.empty(), "the render wrote frames and named no 'artifacts.frames' leaf"); - const std::vector decodes = RecordsNamed(table, "decode.video"); + const std::vector decodes = RecordsNamed(table, "decode.video", render); REQUIRE(!decodes.empty()); for (const nlohmann::json& w : writes) { const double ws = w["start_seconds"].get(); @@ -3236,27 +3310,25 @@ void CheckWriterIsBesideTheDecode(const nlohmann::json& table) { } } -} // namespace - -TEST_CASE("ltx2 video: the three carrying phases contain their work and the load keeps its order") { - Workspace ws; - // `max_phase = 0` is a LOAD extra and is what keeps this on one recipe phase: - // the default `distilled_two_stage` kind would need the latent spatial - // upsampler for its second phase and refuse without one. - vllm::multimodal::VideoModelParams mp = FixtureParams(ws.paths); - mp.extras[vllm::multimodal::kLtx2MaxPhaseExtra] = "0"; - const std::unique_ptr engine = - vllm::multimodal::LoadVideoEngine(mp); - REQUIRE(engine != nullptr); - vllm::multimodal::Ltx2VideoEngine* ltx2 = - dynamic_cast(engine.get()); - REQUIRE(ltx2 != nullptr); - - const vllm::multimodal::VideoResult result = - engine->Generate(FixtureGen(ws.root + "/attribution")); - REQUIRE(result.frame_count == 9); - REQUIRE_MESSAGE(!result.phase_log_path.empty(), "the render wrote no phase table"); - const nlohmann::json table = nlohmann::json::parse(ReadAll(result.phase_log_path)); +// THE PER-RENDER HALF OF THE CONTAINMENT CASE, for one render of the table. +// +// A free function because the case now runs it TWICE, and that is the repair for +// a fourth fresh review's third finding: every decode-side count it made was 1 +// or 1+1, because the fixture renders nine frames as ONE temporal chunk. The +// count assertion is the one that is not a ratio, and a count demonstrated only +// at N=1 cannot tell "once per unit of work" from "once per render". The second +// call renders 81 frames, which is the smallest request this fixture chunks (the +// MULTI-CHUNK case above derives that number), and `min_chunks` makes a geometry +// that silently stopped chunking a RED rather than a vacuous pass. +// +// `render` is the index the table's `render` field carries. It is threaded +// through every lookup here because two renders now share one table; see +// `RecordsNamed`. +void CheckRenderPhases(const nlohmann::json& table, + const vllm::multimodal::Ltx2ConditioningTrace& trace, + const vllm::multimodal::VideoResult& result, int64_t latent_channels, + int64_t render, int64_t min_chunks, double denoise_min_coverage) { + INFO("render = " << render); // ── WHAT THE RENDER COUNTED, which is not what the instrument counted ────── // @@ -3264,28 +3336,48 @@ TEST_CASE("ltx2 video: the three carrying phases contain their work and the load // incremented inside the shared `Evaluate` lambda every sampler arm goes // through; `video_decode_chunks` is incremented in the driver's streaming sink // beside `rendered_frames`. They are the anchors' denominators below. - const vllm::multimodal::Ltx2ConditioningTrace trace = ltx2->last_conditioning(); REQUIRE(trace.completed); REQUIRE(trace.dit_evaluations > 0); REQUIRE(trace.video_decode_chunks > 0); // AND THE CHUNK COUNT IS RE-DERIVED rather than taken on faith, in the shape // the multi-chunk case above already uses: the tiling algebra the decoder runs - // is public, so the group count for this request is computable here from the - // fixture's own factors. A counter that was incremented in the wrong place - // would agree with the phase table and disagree with this. - { - const vllm::Ltx2ScaleFactors factors{8, 32, 32}; - const vllm::Ltx2TileSizeConfig layout = vllm::Ltx2AutoTileSizeConfig(32, 32, factors); - const int64_t latent_t = (result.frame_count - 1) / factors.time + 1; - const std::vector tiles = - vllm::Ltx2CreateTiles(latent_t, 1, 1, layout, factors); - const int64_t groups = - static_cast(vllm::Ltx2GroupTilesByTemporalSlice(tiles).size()); - CHECK_MESSAGE(trace.video_decode_chunks == groups, - "the render counted " << trace.video_decode_chunks << " decoded chunk(s) and " - << "this request's tiling produces " << groups << " temporal group(s)"); - } + // is public, so the group count for this request is computable here. A counter + // that was incremented in the wrong place would agree with the phase table and + // disagree with this. + // + // EVERY INPUT COMES FROM THE FIXTURE OR FROM THE RESULT, which is the repair + // for the second half of the same finding. This block used to hardcode + // `Ltx2ScaleFactors{8, 32, 32}`, `Ltx2AutoTileSizeConfig(32, 32, ...)` and + // `Ltx2CreateTiles(latent_t, 1, 1, ...)` — three constants copied out of the + // fixture header's comment, which is a re-derivation of the algebra and not of + // the geometry. The factors now come from the fixture's OWN decoder block list + // through the same public helper the tiled decode calls, and the tile layout + // from the size the render reported. A fixture whose blocks changed would move + // both sides together before; now it moves only the engine's. + const vllm::Ltx2ConvVideoDecoderConfig vae_cfg = + ltx2_fixture::ReducedVideoDecoderConfig(latent_channels); + const vllm::Ltx2ScaleFactors factors = + vllm::Ltx2VideoScaleFactorsFromBlocks(vae_cfg.decoder_blocks, vae_cfg.patch_size); + const vllm::Ltx2TileSizeConfig layout = + vllm::Ltx2AutoTileSizeConfig(result.height, result.width, factors); + const int64_t latent_t = (result.frame_count - 1) / factors.time + 1; + const int64_t latent_h = result.height / factors.height; + const int64_t latent_w = result.width / factors.width; + REQUIRE(latent_h >= 1); + REQUIRE(latent_w >= 1); + const std::vector tiles = + vllm::Ltx2CreateTiles(latent_t, latent_h, latent_w, layout, factors); + const int64_t groups = + static_cast(vllm::Ltx2GroupTilesByTemporalSlice(tiles).size()); + REQUIRE_MESSAGE(groups >= min_chunks, + "this request decodes in " << groups << " temporal group(s) and this call " + << "exists to exercise at least " << min_chunks + << ". The counts below would be demonstrated at a chunk count the " + "geometry no longer produces"); + CHECK_MESSAGE(trace.video_decode_chunks == groups, + "the render counted " << trace.video_decode_chunks << " decoded chunk(s) and " + << "this request's tiling produces " << groups << " temporal group(s)"); // (1)-(3) THE THREE PHASES THAT CARRY THIS RENDER, each with its anchor. // @@ -3316,6 +3408,17 @@ TEST_CASE("ltx2 video: the three carrying phases contain their work and the load // loosened on an argument nobody has measured: the (0) record count is what // now ties this name to its work, and a red here has a diagnosis printed // beside it in the MESSAGE above. + // + // AND IT IS A PARAMETER, because the uncovered part is WORK and its share + // depends on the geometry. `denoise.step` wraps the denoiser evaluation; + // the post-process and the Euler or res_2s step sit outside it and scale + // with the latent, while the evaluation the anchor covers scales with it + // too. Measured here: 99.28%, 99.38% and 99.55% at nine frames + // (`latent_t = 2`) against 96.55%, 96.90% and 97.09% at 81 + // (`latent_t = 11`), same binary, same box, three runs each. So the + // nine-frame call passes 0.95 and the 81-frame call 0.90 — six points of + // margin on the measured value in both cases, which is the same rule the + // other thresholds are set by and not a loosening to fit. // * `decode.video.chunk` runs from the leaf's own open to the moment the // decoder hands a chunk back, so the only uncovered part is TWO instrument // boundaries — measured 99.44% — and that is why its threshold is the @@ -3336,22 +3439,25 @@ TEST_CASE("ltx2 video: the three carrying phases contain their work and the load const std::vector carrying = { {"denoise", {"denoise.step"}, - 0.95, + denoise_min_coverage, {}, {trace.dit_evaluations}, - "Ltx2ConditioningTrace::dit_evaluations"}, + "Ltx2ConditioningTrace::dit_evaluations", + render}, {"decode.video", {"decode.video.chunk"}, 0.90, {"artifacts.frames"}, {trace.video_decode_chunks + 1}, - "Ltx2ConditioningTrace::video_decode_chunks + 1, the reopen after the last chunk"}, + "Ltx2ConditioningTrace::video_decode_chunks + 1, the reopen after the last chunk", + render}, {"decode.audio", {"decode.audio.mel", "decode.audio.vocoder"}, 0.99, {}, {1, 1}, - "the two calls #1010 names, once each per render"}, + "the two calls #1010 names, once each per render", + render}, // AND THE WRITER, which shipped with no anchor at all and is the leaf the // `decode.video` number is protected by. `artifacts.frames.ppm` wraps the // `WriteFileBytes` loop, so it moves with the WRITES: a mutation that @@ -3362,14 +3468,24 @@ TEST_CASE("ltx2 video: the three carrying phases contain their work and the load // 0.50 is the threshold and it is loose ON PURPOSE. This leaf is about // 0.2 ms on the fixture and holds nine scope boundaries of its own, so the // uncovered part is instrument cost against a sub-millisecond leaf, which - // is the regime the `decode.video` note above already explains. What binds - // here is the containment and the count, not the fraction. + // is the regime the `decode.video` note above already explains. + // + // WHAT BINDS HERE IS THE COUNT AND THE FACT THAT THE WRITER IS NOT + // NESTED, and this note used to say "the containment and the count", which + // a fourth fresh review showed is false. A count of one plus containment + // permits this leaf to be TWICE its anchor and still pass: the leaf may + // grow over any adjacent time nobody named, up to its coverage slack. It + // is harmless on this fixture only because nothing adjacent to the writer + // is stealable — `decode.video` is a declared partner that the `nested` + // assertion and `CheckWriterIsBesideTheDecode` both hold — and that is a + // property of the fixture rather than of this threshold. {"artifacts.frames", {"artifacts.frames.ppm"}, 0.50, {"decode.video"}, {trace.video_decode_chunks}, - "Ltx2ConditioningTrace::video_decode_chunks, one write callback per chunk"}, + "Ltx2ConditioningTrace::video_decode_chunks, one write callback per chunk", + render}, }; for (const Carrying& c : carrying) CheckCarryingPhase(table, c); @@ -3384,19 +3500,50 @@ TEST_CASE("ltx2 video: the three carrying phases contain their work and the load // i.e. around the tile decode itself, so both of its ends are production // events. One record per temporal group, which is one per chunk, and each one // inside the chunk window that produced it. + // + // AND IT HAS A COVERAGE FLOOR, which is the repair for a fourth fresh review's + // second finding. Until this line the vae anchor was checked for cardinality, + // for `nested` and for containment in a chunk window, and its DURATION was + // compared against nothing: `Carrying{"decode.video", ...}` lists only + // `decode.video.chunk` in `parts`, so the vae contributed ZERO to any + // coverage. Moving the scope off `AccumulateTemporalGroup` and onto the + // `buffer.Allocate` beside it — seven lines — left the table reporting + // `decode.video.vae = 0.000 s` beside a five-millisecond `decode.video`, with + // the tile decode inside no sub-scope at all, and both cases in this file + // green. The claim that this leaf "finally has a sub-scope whose ends are both + // production events" was true of the source and unheld by the gate. + // + // THE DENOMINATOR IS `decode.video.chunk`, not the leaf: the leaf's last chunk + // record is the empty window between the final chunk and the end of the + // decode, and both are inside the same leaf, so the chunk windows are the + // tightest thing the vae can be measured against without a second threshold + // for the reopen. Measured at 91.1%, 93.6%, 97.4%, 98.2%, 98.5% and 98.6% of + // the chunk seconds across six renders here — the low end is the nine-frame + // render, whose single chunk is about a millisecond. The floor is 0.50 + // because what is NOT covered is the per-group + // allocation, the overlap blend and the emit — real work whose share of a + // chunk grows as the tile shrinks — plus two instrument boundaries per group + // against a chunk of about a millisecond on this fixture. It is a floor on + // "the tile decode is where this leaf's seconds are", not on the fraction. { - const std::vector vae = RecordsNamed(table, "decode.video.vae"); + const std::vector vae = RecordsNamed(table, "decode.video.vae", render); CHECK_MESSAGE(static_cast(vae.size()) == trace.video_decode_chunks, "the tiled decode emitted " << vae.size() << " 'decode.video.vae' record(s) " << "against " << trace.video_decode_chunks << " chunk(s) the render " << "counted"); - const std::vector chunks = RecordsNamed(table, "decode.video.chunk"); + const std::vector chunks = RecordsNamed(table, "decode.video.chunk", render); + double vae_seconds = 0.0; + double chunk_seconds = 0.0; + for (const nlohmann::json& ch : chunks) { + chunk_seconds += ch["end_seconds"].get() - ch["start_seconds"].get(); + } for (const nlohmann::json& v : vae) { CHECK_MESSAGE(v.value("nested", false), "'decode.video.vae' is not marked nested, so the tile decode is being SUMMED " "as well as the leaf that contains it"); const double vs = v["start_seconds"].get(); const double ve = v["end_seconds"].get(); + vae_seconds += ve - vs; bool inside = false; for (const nlohmann::json& ch : chunks) { if (vs >= ch["start_seconds"].get() - 1e-9 && @@ -3407,7 +3554,114 @@ TEST_CASE("ltx2 video: the three carrying phases contain their work and the load CHECK_MESSAGE(inside, "a 'decode.video.vae' record runs [" << vs << ", " << ve << "] and no 'decode.video.chunk' window encloses it"); } + REQUIRE(chunk_seconds > 0.0); + const double kVaeMinCoverage = 0.50; + MESSAGE(" decode.video.vae = " << vae_seconds << "s over " << vae.size() + << " record(s), of " << chunk_seconds + << "s of decode.video.chunk (" + << (100.0 * vae_seconds / chunk_seconds) << "%)"); + CHECK_MESSAGE(vae_seconds >= kVaeMinCoverage * chunk_seconds, + "'decode.video.vae' covers " << vae_seconds << "s of the " << chunk_seconds + << "s this render spent inside 'decode.video.chunk', under the " + << (100.0 * kVaeMinCoverage) + << "% the tile decode is expected to reach. The anchor is somewhere the " + "work is not, and the leaf W5 ranks its levers from is held by a " + "cardinality alone"); + } + + CheckWriterIsBesideTheDecode(table, render); + + // (5) AND THE CARRYING LEAVES DO NOT OVERLAP. `decode.video` and + // `artifacts.frames` ALTERNATE, because the decoder streams chunks into the + // writer's callback; `decode.audio` follows both; `denoise` precedes all of + // them and is in this list because the review found it was in NO list — the + // loop that shipped covered the decode and the writer and left the largest + // phase of the render out of it. Any overlap would mean a leaf was left open + // across work it does not name, and it would double count into + // `sum_leaf_seconds`. + std::vector> spans; + for (const std::string name : + {"denoise", "decode.video", "artifacts.frames", "decode.audio"}) { + for (const nlohmann::json& r : RecordsNamed(table, name, render)) { + spans.push_back({r["start_seconds"].get(), r["end_seconds"].get()}); + } + } + std::sort(spans.begin(), spans.end()); + for (size_t i = 1; i < spans.size(); ++i) { + CHECK_MESSAGE(spans[i].first >= spans[i - 1].second - 1e-9, + "two of the carrying/writer leaves overlap: [" << spans[i - 1].first << ", " + << spans[i - 1].second << "] and [" << spans[i].first << ", " + << spans[i].second << "]"); } +} + +} // namespace + +TEST_CASE("ltx2 video: the three carrying phases contain their work and the load keeps its order") { + Workspace ws; + // `max_phase = 0` is a LOAD extra and is what keeps this on one recipe phase: + // the default `distilled_two_stage` kind would need the latent spatial + // upsampler for its second phase and refuse without one. + vllm::multimodal::VideoModelParams mp = FixtureParams(ws.paths); + mp.extras[vllm::multimodal::kLtx2MaxPhaseExtra] = "0"; + const std::unique_ptr engine = + vllm::multimodal::LoadVideoEngine(mp); + REQUIRE(engine != nullptr); + vllm::multimodal::Ltx2VideoEngine* ltx2 = + dynamic_cast(engine.get()); + REQUIRE(ltx2 != nullptr); + // The fixture writes its video VAE from `ReducedVideoDecoderConfig(dit.in_channels)` + // (`ltx2_video_fixture.h`), and `CheckRenderPhases` re-derives the tiling from + // that same config rather than from constants. + const int64_t latent_channels = ltx2->dit_params().in_channels; + + // ── RENDER 1: the nine-frame fixture, which decodes in ONE temporal chunk ── + const vllm::multimodal::VideoResult result = + engine->Generate(FixtureGen(ws.root + "/attribution")); + REQUIRE(result.frame_count == 9); + REQUIRE_MESSAGE(!result.phase_log_path.empty(), "the render wrote no phase table"); + const nlohmann::json first = nlohmann::json::parse(ReadAll(result.phase_log_path)); + const int64_t render_one = LastRender(first); + REQUIRE(render_one > 0); + // TAKEN BEFORE THE SECOND GENERATION OVERWRITES IT: `im.trace` is reset on + // every `Generate`, which is half of why the counts here need a render filter. + const vllm::multimodal::Ltx2ConditioningTrace trace_one = ltx2->last_conditioning(); + CheckRenderPhases(first, trace_one, result, latent_channels, render_one, /*min_chunks=*/1, + /*denoise_min_coverage=*/0.95); + + // ── RENDER 2: 81 frames, which CHUNKS, and every count above is checked at + // N > 1 ─────────────────────────────────────────────────────────────────── + // + // WHY A SECOND RENDER. Every decode-side count in the first one is 1 or 1+1 — + // `video_decode_chunks == 1`, `decode.video.chunk == 2`, `artifacts.frames.ppm + // == 1`, `decode.video.vae == 1` — and the re-derivation is `1 == 1`. A count + // demonstrated only at one cannot tell "once per unit of work" from "once per + // render", which is exactly the distinction assertion (0) exists to make. 81 + // frames is the smallest request this fixture chunks; the MULTI-CHUNK case + // above derives that number and asserts it, and `min_chunks` below makes a + // geometry that stopped chunking a red rather than a quiet regression to N=1. + // + // AND IT IS WHAT MAKES THE `render` FILTER LOAD-BEARING. `PhaseLog` resets at + // LOAD, so this second table holds the load AND both renders. Every count + // below is per render and would otherwise be the sum of two. + vllm::multimodal::VideoGenParams multi = FixtureGen(ws.root + "/attribution_multichunk"); + multi.num_frames = 81; + const vllm::multimodal::VideoResult multi_result = engine->Generate(multi); + REQUIRE(multi_result.frame_count == 81); + REQUIRE_MESSAGE(!multi_result.phase_log_path.empty(), "the render wrote no phase table"); + const nlohmann::json table = nlohmann::json::parse(ReadAll(multi_result.phase_log_path)); + const int64_t render_two = LastRender(table); + CHECK_MESSAGE(render_two == render_one + 1, + "the second generation carries render index " << render_two << " and the first " + << render_one << "; the table's `render` field is not the per-generation " + "slice the counts below are taken over"); + CheckRenderPhases(table, ltx2->last_conditioning(), multi_result, latent_channels, render_two, + /*min_chunks=*/2, /*denoise_min_coverage=*/0.90); + + // ...and the FIRST render's records are still in the second render's table and + // still hold, which is what says the filter above selects rather than hides. + CheckRenderPhases(table, trace_one, result, latent_channels, render_one, /*min_chunks=*/1, + /*denoise_min_coverage=*/0.95); // (3c) AND NOTHING BUT AN ANCHOR IS NESTED. The assertion that sees a leaf // swallow a NEIGHBOUR, which the four above cannot: see the note on @@ -3415,7 +3669,6 @@ TEST_CASE("ltx2 video: the three carrying phases contain their work and the load CheckOnlyAnchorsAreNested(table, {"denoise.step", "decode.video.chunk", "decode.video.vae", "decode.audio.mel", "decode.audio.vocoder", "artifacts.frames.ppm"}); - CheckWriterIsBesideTheDecode(table); // (4) THE FLOOR, read as "this name is not detached" and nothing more. See the // note above this case for why it is not tightened toward the measured share. @@ -3437,29 +3690,6 @@ TEST_CASE("ltx2 video: the three carrying phases contain their work and the load "with the work somewhere else"); } - // (5) AND THE CARRYING LEAVES DO NOT OVERLAP. `decode.video` and - // `artifacts.frames` ALTERNATE, because the decoder streams chunks into the - // writer's callback; `decode.audio` follows both; `denoise` precedes all of - // them and is in this list because the review found it was in NO list — the - // loop that shipped covered the decode and the writer and left the largest - // phase of the render out of it. Any overlap would mean a leaf was left open - // across work it does not name, and it would double count into - // `sum_leaf_seconds`. - std::vector> spans; - for (const std::string name : - {"denoise", "decode.video", "artifacts.frames", "decode.audio"}) { - for (const nlohmann::json& r : RecordsNamed(table, name)) { - spans.push_back({r["start_seconds"].get(), r["end_seconds"].get()}); - } - } - std::sort(spans.begin(), spans.end()); - for (size_t i = 1; i < spans.size(); ++i) { - CHECK_MESSAGE(spans[i].first >= spans[i - 1].second - 1e-9, - "two of the carrying/writer leaves overlap: [" << spans[i - 1].first << ", " - << spans[i - 1].second << "] and [" << spans[i].first << ", " - << spans[i].second << "]"); - } - // (6) THE LOAD LEAVES SIT IN THE DRIVER'S OWN ORDER, and this is the weakest // assertion in the case as well as the only one the load has. // @@ -3492,7 +3722,7 @@ TEST_CASE("ltx2 video: the three carrying phases contain their work and the load std::string previous_name; int64_t seen = 0; for (const std::string& name : load_order) { - const std::vector found = RecordsNamed(table, name); + const std::vector found = RecordsNamed(table, name, 0); // A load phase that did not run emits nothing; this fixture carries no // upsampler and no text encoder. if (found.empty()) continue; From 6447d55ac131843e7e51979d516b89523e526dd9 Mon Sep 17 00:00:00 2001 From: Ettore Di Giacinto Date: Thu, 20 Aug 2026 07:26:00 +0000 Subject: [PATCH 06/11] wip(LTX25-RESIDENCY-W0): checkpoint the N1/N6 anchor repairs before the session limit The implementer was interrupted at the rebuild-and-verify step. Committing so the work survives; the ten-mutation regression and the checkers have NOT been re-run on this tree yet, and the next commit owes both. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code] --- .agents/issue-index.md | 1 + .agents/specs/ltx25-device-residency.md | 177 +++++++++++++++++- docs/USAGE.md | 25 ++- .../models/ltx2_video_vae_tiled.cpp | 10 + src/vllm/multimodal/ltx2_video.cpp | 9 + tests/vllm/multimodal/test_ltx2_video.cpp | 34 +++- 6 files changed, 248 insertions(+), 8 deletions(-) diff --git a/.agents/issue-index.md b/.agents/issue-index.md index 85e7ff298..4d7653f20 100644 --- a/.agents/issue-index.md +++ b/.agents/issue-index.md @@ -473,3 +473,4 @@ rather than merged. `scripts/check-agent-record.py` gates both. | [#1435](https://github.com/mudler/vllm.cpp/issues/1435) | `VT-MATMUL-FP8-BLOCK-CUDA` | **A CUDA build without CUTLASS headers segfaults on the block-wise FP8 path instead of refusing by name, and configure reports the feature ENABLED anyway.** Measured 2026-08-20 on `dgx:gpu0` (GB10, cc 12.1) in an `rc` lease at `63d87805c`. `src/vt/cuda/cuda_matmul_fp8_block_cutlass.cu` is the only TU that registers `kMatmulFp8BlockScaled` for `kCUDA` and it reaches `target_sources(vllm PRIVATE ... ${_FP8_CUTLASS_SOURCES})` only through TWO nested conditions in `CMakeLists.txt` -- the outer `if(VLLM_CPP_CUTLASS)` (one occurrence in the file) and, inside it, the `if(VT_CUTLASS_FP8_ARCHS)` that SETS `_FP8_CUTLASS_SOURCES` -- so with no CUTLASS headers the op is unregistered, a device tensor reaches the portable HOST kernel, and the process SIGSEGVs while the reference tier prints `correct but slow`. Anchored by SYMBOL rather than by line on purpose, and the reason is measured on this very row: the guard sat at `CMakeLists.txt:1695@63d87805c` -- the only line number in this row, sha-qualified so it stays resolvable -- and by the time this row was written that number was already stale, then rotted AGAIN by four lines when `origin/main` was merged into the row's own branch mid-review. Two rots inside one pull request. A bare head-relative number here would be permanent, because this index is append-only and no checker scans it, and it would resolve to plausible unrelated code rather than to nothing: at `63d87805c` that line was the guard, and two trees later it was an NCCL include directory. Resolve the anchors above by grepping the symbols; do not trust any number quoted for them. Reachable on a default build: `VLLM_CPP_CUTLASS_FETCH` defaults OFF (the single `option(VLLM_CPP_CUTLASS_FETCH ...)` line, `OFF` at its default argument) and CUTLASS is not a submodule. This contradicts the design note in `cuda_matmul_fp8_block_cutlass.cu`, which states an unsupported build `keeps refusing by name -- which is the honest answer and not the #960/#844 fall-through`: the refusal IS real on the model path (`RefuseUnrunnableQwen3_5DenseFp8Block`, plus the two `VT_CHECK`s) but a caller entering through the op registry, which is what the ported upstream test does, bypasses all three. Second, smaller defect in the same run: `vt_cuda_report_feature` printed `CUDA feature cutlass-fp8: ENABLED for [121a]` for a build whose `build.ninja` contains ZERO references to either FP8 CUTLASS TU, because the report runs on the arch intersection alone, before CUTLASS-header detection. `cmake/CudaArchFeatures.cmake` opens by naming this exact class as what the feature table exists to prevent. NOT fixed in flow: both are code changes on a CUDA path needing their own row, spec and hardware re-gate. Owned by row `VT-MATMUL-FP8-BLOCK-CUDA`, under `## Owed` in [`vt-matmul-fp8-block-cuda.md`](specs/vt-matmul-fp8-block-cuda.md) | bug | | [#1437](https://github.com/mudler/vllm.cpp/issues/1437) | `VT-MATMUL-FP8-BLOCK-CUDA` | **The mainloop-scaled CUTLASS block-wise FP8 GEMM throws `cutlass Invalid status` on upstream's own ported case, on the arch it targets.** FIRST on-hardware execution of #1189 M5 (`489a9a4c0`), measured 2026-08-20 on `dgx:gpu0` (GB10, driver 580.173.02, compute capability 12.1) in an `rc` lease at `63d87805c`, CUDA 13.0 Release, `-DVLLM_CPP_CUDA_ARCHITECTURES=121a -DVLLM_CPP_CUTLASS_FETCH=ON`. G2 (upstream's `test_w8a8_block_fp8_cutlass_matmul` ported whole: M=32, N=576, K=7168, block [128,128], bf16 out) and G7 both THROW from `gemm_op.can_implement(args)`, so CUTLASS rejects the configuration before any launch -- not a numerical disagreement and not a launch failure. G6, G8 and G9 PASS. That this was an EXECUTION and not a skip rests on the throw TEXT, not on a count: `vt cuda: matmul_fp8_block_scaled: cutlass Invalid status` is assembled by the `VT_CUTLASS_CHECK` macro in `src/vt/cuda/cuda_matmul_fp8_block_cutlass.cu`, and the message prefix `matmul_fp8_block_scaled: cutlass ` occurs in NO other translation unit (the per-tensor sibling `cuda_matmul_fp8_cutlass.cu` defines a same-named macro but writes a different message), in a TU compiled only when CUTLASS is found, so no host fallback can produce that string; the run also logged ZERO `[vt reference-tier]` lines, and artifact provenance was confirmed by `cuobjdump --list-elf` reporting `cuda_matmul_fp8_block_cutlass.cu.1.sm_121a.cubin`. Suite: 5 cases, 3 passed, 2 failed, **34 assertions, 0 failed** -- recorded, but attributed rather than leaned on, because **27** of the 34 are G6, which has no `HasCuda()` guard and prints identically on a GPU-less box (8 grid entries x 3 `CHECK` + 3 tile-config `CHECK`s); the device-only remainder is G8's 2 and G9's 5, and a pure skip of this file prints 27. NO shape has had its output compared with the CPU reference: G2 and G7 are the only two cases that make that comparison and BOTH threw before their first assertion -- G7 aborted at `Grid()[0]`, the same M=32 N=576 K=7168 case, so its other seven shapes, including the second ragged-N entry M=8 N=576 K=1024, were never attempted -- and G8 compares a device f32 out against a device bf16 out, which is the kernel against ITSELF. Hypothesis recorded in the issue and explicitly NOT isolated: N=576 is 4*128+64, a ragged final scale block, which the test header records as upstream's reason for choosing it; G8 passing makes the failure shape-dependent rather than universal. This moves the row from `unmeasured` to `measured and failing`, a worse position than the spec previously described. NO speed claim: the lease took no clock control and recorded no contention. Owned by row `VT-MATMUL-FP8-BLOCK-CUDA`, under `## Owed` in [`vt-matmul-fp8-block-cuda.md`](specs/vt-matmul-fp8-block-cuda.md) | measurement | | [#1440](https://github.com/mudler/vllm.cpp/issues/1440) | `LTX25-DEVICE-RESIDENCY` | **W0's phase-table gate cannot see an anchor that moves WITH its leaf.** Found by the third fresh review of [#1408](https://github.com/mudler/vllm.cpp/pull/1408) against `89261c955`. The containment case asserts containment, coverage, exclusivity and non-overlap for each carrying phase, and all four are RATIOS against the sub-scope anchor, so a defect that moves the leaf and its anchor together satisfies every one of them. M12 emits `denoise.step` for step 0 only, closes `denoise` there and opens `phase.finish` over steps 1-7: **exit 0, 106/106 and 350/350**, over a table putting `phase.finish` at 28.73% against a `denoise` of 6.11% on a run whose honest denoise was 39.3% -- 82% of the largest phase re-labelled, which is M7's defect on M7's phase through the gate M7 produced. M11 leaves the `write(2)`s inside `decode.video` and emits `artifacts.frames` covering nothing: **exit 0, 124/124 and 422/422**, coverage 99.68%, with `artifacts.frames` reporting 4 microseconds for nine PPM files and the whole writer charged to the leaf W5's lever is measured from. Three smaller findings beside them: the case was titled 'each named phase CONTAINS the work it is named after' and checked three of about fourteen, with `load.dit` -- the phase W2 and W3 both act on, ~7.5 minutes of DiT staging on the shipped 21B -- unanchored and its name swappable with `load.prompt_embeds` for 96% of the load's seconds with every gate green; `### Owed out of W0` named seven sub-millisecond leaves and omitted `load.*`, `conditioning.tower` and `conditioning.connector`, the last two being #1269's 39-100% bound that W4 acts on; and the `denoise` coverage margin was justified as 'a percent of a 40 ms leaf' when the per-boundary cost is FIXED while the leaf shrinks, measured 99.228% at a 20 ms leaf against a 0.95 threshold, which is a false-RED risk on faster hardware rather than a false pass. NOT a correctness defect in the instrument, which all three reviews found reachable on the shipped default. Fixed in flow. Spec [`ltx25-device-residency.md`](specs/ltx25-device-residency.md) `### What a THIRD fresh review found` | bug | +| [#1446](https://github.com/mudler/vllm.cpp/issues/1446) | `LTX25-DEVICE-RESIDENCY` | **W0's phase-table gate cannot see two SIBLING anchor names exchanged, and `decode.video.vae`'s placement is gated by a cardinality alone.** Found by the FOURTH fresh review of [#1441](https://github.com/mudler/vllm.cpp/pull/1441) against `165db635c`, where every earlier mutation (M12, M11, M11b, M13, M7, M4, M10) is already RED. **N1** swaps the `decode.audio.mel` and `decode.audio.vocoder` string literals in `src/vllm/multimodal/ltx2_video.cpp` (+2/-2, compile rc 0): containment **exit 0, 1/1, 194/194** and SUMS **exit 0, 1/1, 446/446**, byte-identical to the honest claim, over a table reading `decode.audio.vocoder 0.007 s` / `decode.audio.mel 0.220 s` where the honest table has them reversed -- **96.8% of `decode.audio`'s decomposed seconds on the wrong name**, on a leaf that is 47.8% of that render's named leaf seconds, and [#1010](https://github.com/mudler/vllm.cpp/issues/1010) asks for the vocoder BY NAME. Nothing in the case tied a sub-scope NAME to a POSITION: the counts are `{1, 1}` either way, containment holds for both, `subs` is sorted by start before the overlap check, and `CheckOnlyAnchorsAreNested` is a set-membership test -- and the load-order assertion in that same change exists BECAUSE M13 showed a sibling swap moves 96% with everything green. **N6** moves `decode.video.vae` off `AccumulateTemporalGroup` onto the `buffer.Allocate` beside it in `src/vllm/model_executor/models/ltx2_video_vae_tiled.cpp` (+7/-8, compile rc 0): containment **exit 0, 194/194**, SUMS **exit 0, 446/446**, with the table reporting `decode.video.vae = 0.000 s` beside a five-millisecond `decode.video` and the tile decode inside no sub-scope, because that anchor was checked for cardinality, `nested` and containment and its DURATION was compared against nothing -- `Carrying{"decode.video", ...}` lists only `decode.video.chunk` in `parts`. Three smaller findings beside them: every decode-side count was demonstrated at **N = 1**, which cannot distinguish 'once per unit of work' from 'once per render'; the counts compared a per-render counter (`im.trace`, reset each `Generate`) against a process-wide record count (`PhaseLog`, reset at load); and a leaf may grow over adjacent UNNAMED time up to its coverage slack (about 5.3% `denoise`, 11% `decode.video`, 1% `decode.audio`, up to 100% `artifacts.frames`), which the `nested` invariant cannot see because that invariant is about neighbours. NOT a correctness defect in the instrument, which four reviews have found reachable on the shipped default. FIXED IN FLOW: assertion (1b) asserts sibling order, assertion (6) gives the vae a coverage floor against `decode.video.chunk` seconds (measured 91.1%-98.6%, floored at 0.50), the case renders a second time at 81 frames so every count is demonstrated at N > 1, and `RecordsNamed` filters on the `render` field. Spec [`ltx25-device-residency.md`](specs/ltx25-device-residency.md) `### What a FOURTH fresh review found` | bug | diff --git a/.agents/specs/ltx25-device-residency.md b/.agents/specs/ltx25-device-residency.md index 9341ab893..3689967a0 100644 --- a/.agents/specs/ltx25-device-residency.md +++ b/.agents/specs/ltx25-device-residency.md @@ -1125,14 +1125,24 @@ nested record. beside it, and `artifacts.frames` joins the carrying phases with its record count taken from `video_decode_chunks`. Its coverage threshold is 0.50 and loose on purpose: the leaf is 0.2 ms on this fixture and carries its own boundaries, so -what binds is the containment and the count. Measured 97.97% honest. +what binds is the COUNT and the fact that the writer is not `nested`. Measured +97.97% honest. **This sentence used to say "the containment and the count", and +a fourth fresh review showed that is false:** a count of one plus containment +permits the leaf to be TWICE its anchor, because a leaf may grow over adjacent +time nobody named up to its coverage slack. It is harmless on this fixture only +because nothing adjacent to the writer is stealable — `decode.video` is a +declared partner that the `nested` assertion and `CheckWriterIsBesideTheDecode` +both hold — which is a property of the fixture and not of the threshold. *And `decode.video` got the VAE-side sub-scope this spec recorded as owed.* `decode.video.vae` opens inside `Ltx2ConvVideoDecodeTiled` around `AccumulateTemporalGroup`, one record per temporal group, so that leaf finally has a sub-scope whose ends are both production events rather than statements adjacent to its own `Open`. Authority for that one file was extended by the -operator for exactly this. +operator for exactly this. **As written at `165db635c` that was a claim about the +SOURCE that no gate held**, and the fourth review below is what closed it: the +anchor was checked for cardinality, for `nested` and for containment in a chunk +window, and its duration was compared against nothing at all. *And the load got the only check available inside this stage's authority: its ORDER.* An anchor for `load.dit` would have to open inside @@ -1177,6 +1187,142 @@ than none. The phase table is **v23**, and the `>= 22` floor in moved with it. The table also gained the v22 row for `mmproj_path` it was missing. +### What a FOURTH fresh review found + +Tracked by [#1446](https://github.com/mudler/vllm.cpp/issues/1446), filed and +fixed in the same flow. + +The fourth reviewer called the third repair "by far the strongest iteration" and +re-ran every earlier mutation against it: M12, M11, M11b, M13, M7, M4 and M10 are +all RED, and the record count is the instrument that does it. **Two of the +repair's own stated guarantees still fell to two-line mutations, and both are on +the anchored set the change declares.** + +**N1 — sibling anchor names are swappable.** Swap the `decode.audio.mel` and +`decode.audio.vocoder` string literals in the driver (1 file, +2/-2, compile +rc 0). Both gates were green: the containment case **exit 0, 1/1, 194/194** and +the SUMS case **exit 0, 1/1, 446/446**, byte-identical to the honest claim, over +a table reading `decode.audio.vocoder 0.007 s` / `decode.audio.mel 0.220 s` where +the honest table has them reversed. **96.8% of `decode.audio`'s decomposed +seconds land on the wrong name**, on a leaf that is 47.8% of that render's named +leaf seconds and was 25.5% of wall in the first artifact this row shipped, and +[#1010](https://github.com/mudler/vllm.cpp/issues/1010) asks for the vocoder BY +NAME. W1 would have ranked the wrong model. + +Every assertion held because **none tied a sub-scope name to a position**: the +counts are `{1, 1}` either way, containment holds for both, `subs` is sorted by +start before the overlap check, and `CheckOnlyAnchorsAreNested` is a +set-membership test. The sharp part is that the third repair added the load-order +assertion (6) *because* M13 showed a sibling name swap moves 96% with everything +green, and then left the identical hole on the one sibling pair inside its own +anchored set. + +**N6 — `decode.video.vae`'s placement was not gated at all.** Move the scope off +`AccumulateTemporalGroup` and onto the `buffer.Allocate` in the same loop body +(1 file, +7/-8, compile rc 0): containment **exit 0, 194/194**, SUMS **exit 0, +446/446**. The table then reports `decode.video.vae = 0.000 s` beside a +five-millisecond `decode.video`, and the tile decode — the work the anchor is +named for — sits inside no sub-scope. The gate checked that anchor for +cardinality, `nested` and containment in a chunk window and nothing else; +`Carrying{"decode.video", ...}` lists only `decode.video.chunk` in `parts`, so +the vae contributed **zero** to coverage. The anchor also fired in neither M11 +nor M11a, so on the evidence it constrained nothing the gate did not already +hold, and the W5 `## Owed` item it was written to pay was paid in prose only. + +Three smaller findings beside them. + +**N-count — every decode-side count was demonstrated at N = 1.** The fixture +renders one chunk, so `video_decode_chunks == 1`, `decode.video.chunk == 2`, +`artifacts.frames.ppm == 1`, `decode.video.vae == 1` and the re-derivation +`trace.video_decode_chunks == groups` is `1 == 1`. A count at one cannot +distinguish "once per unit of work" from "once per render", which is exactly the +distinction assertion (0) exists to make; only `denoise.step` was shown at N = 8. +The re-derivation also hardcoded `Ltx2ScaleFactors{8, 32, 32}`, +`Ltx2AutoTileSizeConfig(32, 32, ...)` and `Ltx2CreateTiles(latent_t, 1, 1, ...)` +rather than reading the fixture's own config — safe, because it can only +false-red, but not the independent derivation the pull-request body implied. + +**N-render — a per-render counter was compared against a process-wide record +count.** `im.trace` resets on every `Generate` and `PhaseLog` resets only at +load, and `RecordsNamed` did not filter on the `render` field the records already +carry. Correct only because the case rendered exactly once. + +**N-slack — a leaf may grow over adjacent UNNAMED time.** The `nested` invariant +sees a leaf swallow a NEIGHBOUR; nothing sees it swallow `unaccounted_seconds`. +The room is each leaf's coverage slack: about 5.3% for `denoise`, 11% for +`decode.video`, 1% for `decode.audio` and up to 100% for `artifacts.frames`. Not +a defect in the repair — it is disclosed here quantitatively rather than closed, +because closing it needs a threshold this box cannot measure. + +**What closes N1 and N6.** + +*Assertion (1b), THE SIBLING ORDER.* Two sub-scopes of one leaf must appear in +the order `parts` lists them, which is the order the driver runs them, compared +on each name's FIRST record so a part that legitimately repeats +(`decode.video.chunk`) is unaffected and a leaf with one part name is +unconstrained. It is the same shape assertion (6) already uses for the load, and +it proves that a swapped PAIR is a red — not that either name covers the call it +is named after. That needs a scope inside `Ltx2AudioDecoderForward` / +`Ltx2VocoderWithBweForward` and is recorded below beside `load.dit`. + +*Assertion (6), THE VAE COVERAGE FLOOR.* `decode.video.vae` must cover at least +half of the render's `decode.video.chunk` seconds. The denominator is the chunk +windows rather than the leaf, because the leaf's last chunk record is the empty +window between the final chunk and the end of the decode. Measured 91.1%, 93.6%, +97.4%, 98.2%, 98.5% and 98.6% over six renders here; the floor is 0.50 because +the uncovered part is the per-group allocation, the overlap blend and the emit — +real work whose share grows as the tile shrinks — plus two instrument boundaries +against a chunk of about a millisecond on this fixture. + +*And the counts are now demonstrated at N > 1.* The containment case renders +TWICE: nine frames, which decode in one temporal chunk, and 81, which is the +smallest request this fixture chunks and which the MULTI-CHUNK case above already +derives and asserts. Every per-render assertion runs over each render of one +table, `min_chunks` makes a geometry that stopped chunking a red rather than a +quiet regression to N = 1, and the tiling re-derivation now reads the fixture's +own decoder block list through `Ltx2VideoScaleFactorsFromBlocks` and the size the +render reported, in place of the three constants. The second render is also what +makes the new `render` filter load-bearing rather than hypothetical: that table +holds the load and BOTH renders, so an unfiltered count would be the sum of two. + +**One threshold moved, and it is a parameter rather than a loosening.** +`denoise`'s coverage floor is the call's argument now: 0.95 at nine frames and +0.90 at 81. The uncovered part is WORK — the post-process and the Euler or res_2s +step, which no anchor wraps — and its share depends on the geometry. Measured +99.28%, 99.38% and 99.55% at `latent_t = 2` against 96.55%, 96.90% and 97.09% at +`latent_t = 11`, same binary, same box, three runs each. Six points of margin on +the measured value in both cases, which is the rule the other thresholds are set +by. + +| # | Mutation | `git diff --stat` | compile | containment case | SUMS case | +|---|---|---|---|---|---| +| — | honest | (none) | rc 0 | exit 0, 1/1, **527/527** | exit 0, 1/1, **446/446** | +| **N1** | swap the `decode.audio.mel` and `decode.audio.vocoder` scope names — the **sibling-swap** mutation | 1 file, +2/-2 | rc 0 | **RED**, exit 1, 1 case failed, **527 assertions, 3 failed**, on the sibling order in each of the three per-render checks | exit 0, 446/446 | +| **N6** | move `decode.video.vae` off `AccumulateTemporalGroup` onto `buffer.Allocate` — the **anchor beside the work** mutation | 1 file, +5/-9 | rc 0 | **RED**, exit 1, 1 case failed, **527 assertions, 3 failed**, on the vae coverage floor | exit 0, 446/446 | +| M13 | re-anchored: swap the `load.dit` and `load.prompt_embeds` scope names | 1 file, +2/-2 | rc 0 | **RED**, exit 1, 1 case failed, **527 assertions, 2 failed**, on the load order | exit 0, 446/446 | +| M12 | re-anchored: `denoise.step` for step 0 only, `denoise` closed there, `phase.finish` over steps 1-7 | 1 file, +8/-1 | rc 0 | **RED**, exit 1, 1 case failed, **462 assertions, 3 failed** | exit 0, 374/374 | +| M7 | re-anchored: close `denoise` after the first sampler step and open `phase.finish` there | 1 file, +6/-0 | rc 0 | **RED**, exit 1, 1 case failed, **539 assertions, 24 failed** | exit 0, 458/458 | +| M4 | re-anchored: `decode.video` left open across the audio decode | 1 file, +2/-1 | rc 0 | **RED**, exit 1, 1 case failed, **88 assertions, 2 failed** (a `REQUIRE_FALSE` aborts the case early) | exit 0, 446/446 | +| M10 | re-anchored: after a chunk, re-label the decode as the writer | 1 file, +1/-1 | rc 0 | **RED**, exit 1, 1 case failed, **529 assertions, 4 failed** | exit 0, 446/446 | +| M11 | re-anchored: writes stay inside `decode.video`, `artifacts.frames` emitted empty | 1 file, +3/-4 | rc 0 | **RED**, exit 1, 1 case failed, **527 assertions, 4 failed** | exit 0, 446/446 | +| M11b | re-anchored: both `Close`s below the write, writer still around the loop | 1 file, +2/-2 | rc 0 | **RED**, exit 1, 1 case failed, **119 assertions, 1 failed** | exit 0, 446/446 | +| N10 | detach `artifacts.frames.ppm` from the `WriteFileBytes` loop by closing it before the loop runs | 1 file, +1/-1 | rc 0 | **RED**, exit 1, 1 case failed, **527 assertions, 3 failed** | exit 0, 446/446 | + +Every mutation was applied by exact-text replacement that refuses unless the +pattern occurs exactly once, with its `git diff --stat` and its compile return +code printed and the tree restored byte-for-byte and re-verified with +`git status --porcelain` afterwards. A mutation that fails to build, or that +never applies, reads exactly like a passing test. + +**The SUMS column is still not a defect,** for the fourth time: a sum cannot see +a transfer, an anchor that moved with its leaf, or a pair of anchor names that +were exchanged. + +**No timing conclusion is drawn from this box.** Every threshold above is a +within-run share. The leaf sums moved 0.658 s, 0.706 s and 0.779 s across three +runs of one binary at one geometry while this was written, and earlier runs of +the same case on the same box reported 0.080 s and 0.476 s. + ### No claim file, and the checker says why A `CLAIM-LTX25-RESIDENCY-W0.md` was written and then removed. @@ -1226,6 +1372,33 @@ which is where this spec's `### Decisions taken here` already said they would be The `load` and `generate` records are SPANS and are never summed, so they are not in this list; `sum_rule` in every emitted table says which records add up. +* **AND WHAT IS OWED ABOUT THE ANCHORS THEMSELVES**, which this entry was silent + about until a fourth fresh review asked. The list above is honest and complete + for LEAVES; it said the four carrying phases are anchored and stopped there, as + though "anchored" were a single property. It is three, and only two of them + hold: + + | About an anchor | State | + |---|---| + | It runs once per unit of work the RENDER counted | **PROVEN**, at N = 8 for `denoise.step` and at N = 1 and N = 2 for every decode-side count, over two renders in one table. The counters come from `Ltx2ConditioningTrace` and the chunk count is re-derived from `Ltx2GroupTilesByTemporalSlice` | + | Sibling anchors under one leaf are in the driver's own order | **PROVEN** for the one sibling pair that exists (`decode.audio.mel` before `decode.audio.vocoder`), and it is an ORDER, not an identity | + | Each anchor covers the CALL it is named after | **NOT PROVEN, for any of the six.** `denoise.step`, `decode.video.chunk`, `decode.audio.mel`, `decode.audio.vocoder` and `artifacts.frames.ppm` are all opened by a statement ADJACENT to the call. `decode.video.vae` is the only one whose ends are both production events, and what holds its placement is a coverage floor against `decode.video.chunk` — a ratio between two anchors, which is weaker than a count and stronger than the nothing that held it at `165db635c` | + + The third row is the same debt `load.dit` carries and it is not smaller for + being on the render side. Closing it needs a scope inside the callee — + `Ltx2AudioDecoderForward`, `Ltx2VocoderWithBweForward`, + `Ltx2LoadDitFromSafetensors` — which is where the placement stops being a + statement about where somebody wrote a line. +* **A LEAF MAY STILL GROW OVER TIME NOBODY NAMED, and the room is measurable.** + The `nested` invariant sees a leaf swallow a NEIGHBOUR, because the neighbour + turns `nested` and leaves the timeline. Nothing sees a leaf swallow + `unaccounted_seconds`. The room is each leaf's coverage slack: **`denoise` + about 5.3%, `decode.video` about 11%, `decode.audio` about 1%, and + `artifacts.frames` up to 100%** — that last one meaning the writer's leaf may + be twice its anchor and pass, which is harmless on this fixture only because + nothing adjacent to it is stealable. Disclosed rather than closed: a tighter + floor is a claim about a measured share, and the measured share is the quantity + this box destroys. * **F10 IS NOW GATED, and the sentence that said it could not be was wrong.** This entry used to read "F10 is an absence — a thread that keeps running — which no assertion in this suite is positioned to observe", and asserted that a diff --git a/docs/USAGE.md b/docs/USAGE.md index 8bd53f19a..10121bac2 100644 --- a/docs/USAGE.md +++ b/docs/USAGE.md @@ -1102,14 +1102,31 @@ that is already counted, so adding them would make `unaccounted_seconds` the residue of double counting instead of time nobody named. A nested record is also what makes a phase NAME checkable, and it is checkable in -two ways rather than one. A leaf that claims to cover the denoise must ENCLOSE +four ways rather than one. A leaf that claims to cover the denoise must ENCLOSE its own `denoise.step` records; one that stops short of the loop, or that hands -the back half of it to a neighbouring name, no longer does. And the anchor must +the back half of it to a neighbouring name, no longer does. The anchor must appear once per unit of work the RENDER counted — one `denoise.step` per denoiser evaluation, one `decode.video.chunk` per streamed chunk plus the reopen -after the last one — which is the only check here that is not a ratio against the +after the last one — which is the only check that is not a ratio against the leaf, and therefore the only one an anchor that moves WITH its leaf cannot -satisfy. The four phases that carry a render each carry such an anchor. +satisfy. Two SIBLING anchors under one leaf must appear in the order the render +runs them, because nothing else distinguishes them: swapping the +`decode.audio.mel` and `decode.audio.vocoder` names moves 96.8% of that leaf's +decomposed seconds onto the wrong model and changes nothing any ratio can see. +And `decode.video.vae` must cover most of the `decode.video.chunk` seconds, +because a cardinality alone permits the anchor to sit beside the tile decode +rather than on it. The four phases that carry a render each carry such an anchor. + +**What the anchors do NOT prove**, stated here because the table invites the +opposite reading. An anchor proves that the name is where the work is, in the +order the render runs it; it does not prove that a name covers the call it is +named after — that needs a scope inside the callee, and `load.dit` and the two +`conditioning.*` leaves do not have one. A leaf may also grow over adjacent time +that NOBODY named, up to its coverage slack: 5.3% for `denoise`, 11% for +`decode.video`, 1% for `decode.audio` and up to 100% for `artifacts.frames`, +whose threshold is loose because the leaf is sub-millisecond. Growing over a +NEIGHBOUR is caught, because the neighbour turns `nested`; growing over +`unaccounted_seconds` is not. A record that is `nested` and is NOT one of those anchors means a leaf was left open across a phase it does not name: the neighbour turns `nested`, leaves diff --git a/src/vllm/model_executor/models/ltx2_video_vae_tiled.cpp b/src/vllm/model_executor/models/ltx2_video_vae_tiled.cpp index f4bc6d627..0a4b1035e 100644 --- a/src/vllm/model_executor/models/ltx2_video_vae_tiled.cpp +++ b/src/vllm/model_executor/models/ltx2_video_vae_tiled.cpp @@ -249,6 +249,16 @@ void Ltx2ConvVideoDecodeTiled(const Ltx2ConvVideoDecoderConfig& config, // both ends. One record per temporal group, which is one record per chunk // the sink is handed, so the count is a quantity the instrument cannot move. // Nested, so the table's sum does not change. + // + // THE PLACEMENT IS GATED BY A COVERAGE FLOOR AND NOT BY THE COUNT ALONE. + // Moving this scope one statement up, onto `buffer.Allocate`, keeps the + // count, the `nested` flag and the containment in a chunk window — and a + // fourth fresh review measured what that costs: `decode.video.vae = 0.000 s` + // beside a five-millisecond `decode.video`, with the tile decode inside no + // sub-scope, both gates green. `test_ltx2_video`'s containment case now + // requires these records to cover at least half of the render's + // `decode.video.chunk` seconds (measured 91.1%-98.6%), so an anchor that + // sits BESIDE the work rather than ON it is a red. std::vector curr_weights; { const ::vllm::multimodal::phase::Scope vae_phase("decode.video.vae"); diff --git a/src/vllm/multimodal/ltx2_video.cpp b/src/vllm/multimodal/ltx2_video.cpp index ef07fd4cc..72fe469a1 100644 --- a/src/vllm/multimodal/ltx2_video.cpp +++ b/src/vllm/multimodal/ltx2_video.cpp @@ -4547,6 +4547,15 @@ VideoResult Ltx2VideoEngine::Generate(const VideoGenParams& gen) { // `AccumulateTemporalGroup`, so `decode.video` has one sub-scope whose ends // are both production events; and the gate asserts that nothing but an anchor // is emitted NESTED, which is what a swallowed `artifacts.frames` becomes. + // + // AND THE VAE SUB-SCOPE IS HELD BY A COVERAGE FLOOR, not by its existence. A + // fourth fresh review moved that scope off `AccumulateTemporalGroup` and onto + // the `buffer.Allocate` beside it — seven lines — and the table reported + // `decode.video.vae = 0.000 s` beside a five-millisecond `decode.video`, with + // the tile decode inside no sub-scope at all and both gates green, because the + // vae anchor was checked for cardinality, `nested` and containment and its + // DURATION was compared against nothing. The gate now requires it to cover at + // least half of this render's `decode.video.chunk` seconds. size_t chunk_handle = phase::PhaseLog::Instance().Open("decode.video.chunk", /*span=*/false); Ltx2VideoDecodeStreaming( diff --git a/tests/vllm/multimodal/test_ltx2_video.cpp b/tests/vllm/multimodal/test_ltx2_video.cpp index d44963781..c35eeb047 100644 --- a/tests/vllm/multimodal/test_ltx2_video.cpp +++ b/tests/vllm/multimodal/test_ltx2_video.cpp @@ -2925,8 +2925,14 @@ TEST_CASE("ltx2 video: a render through the ABI emits a phase table that SUMS to // measured from — because a swallowed neighbour turns `nested`, leaves the // sum, and overlaps nothing. // -// So the case now asserts SIX things, and the first and the sixth are the ones -// that are not ratios: +// So the case now asserts the list below, over TWO renders. Read it by what each +// item is measured against, because that is what decides which defect it can +// see. (0), (1b) and (5) are measured against nothing the instrument controls — +// a count the render kept, an order the driver runs in, a flag the emitter sets +// — and they are the three that survive an anchor moving with its leaf. (1), +// (2), (3) and (4) are ratios against the anchor. (6) is a ratio between two +// anchors, which is a weaker guarantee than (0) and a stronger one than nothing, +// and nothing was what held it before. // // 0. RECORD COUNT the anchor runs once per unit of work the RENDER counted. // `denoise.step` once per `Ltx2ConditioningTrace:: @@ -2944,6 +2950,15 @@ TEST_CASE("ltx2 video: a render through the ABI emits a phase table that SUMS to // chunk BACK, which is a production event and not an // instrument statement. `decode.audio.mel` and // `decode.audio.vocoder` wrap the two calls #1010 names. +// 1b. SIBLING ORDER two sub-scopes of one leaf appear in the order the driver +// runs them, compared on each name's first record. NOTHING +// ELSE TIES A SUB-SCOPE NAME TO A POSITION — the counts are +// per name, containment holds for either, the intervals are +// sorted before the overlap check, and the nested check is a +// set membership — so swapping the `decode.audio.mel` and +// `decode.audio.vocoder` literals moved 96.8% of that leaf's +// decomposed seconds onto the wrong name with everything +// green. This is what N1 fails. // 2. COVERAGE the sub-scopes together account for nearly all of the leaf, // so a leaf that encloses its own work PLUS a phase nobody // named still fails. The threshold is per phase and is set @@ -2965,6 +2980,21 @@ TEST_CASE("ltx2 video: a render through the ABI emits a phase table that SUMS to // `AccumulateTemporalGroup`, so that leaf has one sub-scope // whose ends are both production events rather than // statements adjacent to its own. +// 6. VAE COVERAGE `decode.video.vae` must cover at least half of this +// render's `decode.video.chunk` seconds. Until this landed +// the vae anchor was held by cardinality, `nested` and +// containment alone and its DURATION was compared against +// nothing, so moving it one statement up onto +// `buffer.Allocate` reported `decode.video.vae = 0.000 s` +// beside a five-millisecond `decode.video` with the tile +// decode inside no sub-scope, both gates green. N6. +// 7. THE COUNTS AT N > 1. The case renders twice — nine frames, which decode +// in one temporal chunk, and 81, which chunk — and runs +// assertions 0 through 6 over each render of one table. +// Every decode-side count was previously 1 or 1+1, which +// cannot distinguish "once per unit of work" from "once per +// render". Both lookups are filtered by the record's +// `render` field; see `RecordsNamed`. // // WHAT WAS TRIED FIRST AND REJECTED, MEASURED RATHER THAN ARGUED. The obvious // repair is a differential: render at two frame counts and require the phases From b4b75af2337589dc741a2c43f35145e0dc48961b Mon Sep 17 00:00:00 2001 From: Ettore Di Giacinto Date: Thu, 20 Aug 2026 08:19:57 +0000 Subject: [PATCH 07/11] record(LTX25-RESIDENCY-W0): the ten-mutation set re-run on the merged head, and the ENOSPC run that was not a verdict MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The mutation table in `### What a FOURTH fresh review found` was measured before `origin/main` was merged, and one of the incoming commits (`d0eff4f25`, W0-LIVE) edits this row's own driver. A mutation proof inherited from an earlier head is a claim about a tree that no longer exists, so all ten were re-applied to the merge commit `8c66ffa16` and the table now carries those numbers. Every one is RED, every one compiled rc 0, and every one restored byte-for-byte with a clean `git status --porcelain`. N6's pattern stopped matching on the way, because this branch's own repair edited the comment above the scope it moves. The harness refused rather than reporting a green, which is the whole reason it counts occurrences: a mutation that never applies reads exactly like a passing test. N10 is recorded as RECONSTRUCTED. The fourth review's own definition of it is not in this tree or on the forge, so the label carries the mutation its name describes — close `artifacts.frames.ppm` before the write loop rather than after it — and the section says so rather than implying provenance it does not have. The four suites on that head are added beside the table, and so is the run that reported three cases THREW: the box filled to 100% and the exceptions read `No space left on device`. That is a full disk presenting as a claim about the code, and the recorded numbers are the re-run with 91 GB free. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code] --- .agents/specs/ltx25-device-residency.md | 37 +++++++++++++++++++++---- 1 file changed, 31 insertions(+), 6 deletions(-) diff --git a/.agents/specs/ltx25-device-residency.md b/.agents/specs/ltx25-device-residency.md index 48fcc5af2..4b576cf80 100644 --- a/.agents/specs/ltx25-device-residency.md +++ b/.agents/specs/ltx25-device-residency.md @@ -1296,20 +1296,33 @@ step, which no anchor wraps — and its share depends on the geometry. Measured the measured value in both cases, which is the rule the other thresholds are set by. +**Re-run on the FINAL head**, the merge commit `8c66ffa16`, and not on the head each mutation was +first measured on: `origin/main` moved twice while this branch was open and one +of the incoming commits (`d0eff4f25`, W0-LIVE) edits this row's own driver, so a +mutation proof inherited from an earlier head is a claim about a tree that no +longer exists. + | # | Mutation | `git diff --stat` | compile | containment case | SUMS case | |---|---|---|---|---|---| -| — | honest | (none) | rc 0 | exit 0, 1/1, **527/527** | exit 0, 1/1, **446/446** | +| — | honest | (none) | rc 0 | exit 0, 1/1 cases, **527/527** | exit 0, 1/1, **446/446** | | **N1** | swap the `decode.audio.mel` and `decode.audio.vocoder` scope names — the **sibling-swap** mutation | 1 file, +2/-2 | rc 0 | **RED**, exit 1, 1 case failed, **527 assertions, 3 failed**, on the sibling order in each of the three per-render checks | exit 0, 446/446 | -| **N6** | move `decode.video.vae` off `AccumulateTemporalGroup` onto `buffer.Allocate` — the **anchor beside the work** mutation | 1 file, +5/-9 | rc 0 | **RED**, exit 1, 1 case failed, **527 assertions, 3 failed**, on the vae coverage floor | exit 0, 446/446 | +| **N6** | move `decode.video.vae` off `AccumulateTemporalGroup` onto `buffer.Allocate` — the **anchor beside the work** mutation | 1 file, +7/-7 | rc 0 | **RED**, exit 1, 1 case failed, **527 assertions, 3 failed**, on the vae coverage floor | exit 0, 446/446 | +| M12 | re-anchored: `denoise.step` for step 0 only, `denoise` closed there, `phase.finish` over steps 1-7 | 1 file, +8/-1 | rc 0 | **RED**, exit 1, 1 case failed, **462 assertions, 5 failed** | exit 0, 374/374 | +| M11 | re-anchored: writes stay inside `decode.video`, `artifacts.frames` emitted empty | 1 file, +3/-4 | rc 0 | **RED**, exit 1, 1 case failed, **527 assertions, 4 failed** | exit 0, 446/446 | +| M11b | re-anchored: both `Close`s below the write, writer still around the loop | 1 file, +2/-2 | rc 0 | **RED**, exit 1, 1 case failed, **119 assertions, 1 failed** | exit 0, 446/446 | | M13 | re-anchored: swap the `load.dit` and `load.prompt_embeds` scope names | 1 file, +2/-2 | rc 0 | **RED**, exit 1, 1 case failed, **527 assertions, 2 failed**, on the load order | exit 0, 446/446 | -| M12 | re-anchored: `denoise.step` for step 0 only, `denoise` closed there, `phase.finish` over steps 1-7 | 1 file, +8/-1 | rc 0 | **RED**, exit 1, 1 case failed, **462 assertions, 3 failed** | exit 0, 374/374 | | M7 | re-anchored: close `denoise` after the first sampler step and open `phase.finish` there | 1 file, +6/-0 | rc 0 | **RED**, exit 1, 1 case failed, **539 assertions, 24 failed** | exit 0, 458/458 | | M4 | re-anchored: `decode.video` left open across the audio decode | 1 file, +2/-1 | rc 0 | **RED**, exit 1, 1 case failed, **88 assertions, 2 failed** (a `REQUIRE_FALSE` aborts the case early) | exit 0, 446/446 | -| M10 | re-anchored: after a chunk, re-label the decode as the writer | 1 file, +1/-1 | rc 0 | **RED**, exit 1, 1 case failed, **529 assertions, 4 failed** | exit 0, 446/446 | -| M11 | re-anchored: writes stay inside `decode.video`, `artifacts.frames` emitted empty | 1 file, +3/-4 | rc 0 | **RED**, exit 1, 1 case failed, **527 assertions, 4 failed** | exit 0, 446/446 | -| M11b | re-anchored: both `Close`s below the write, writer still around the loop | 1 file, +2/-2 | rc 0 | **RED**, exit 1, 1 case failed, **119 assertions, 1 failed** | exit 0, 446/446 | +| M10 | re-anchored: after a chunk, re-label the decode as the writer | 1 file, +1/-1 | rc 0 | **RED**, exit 1, 1 case failed, **529 assertions, 5 failed** | exit 0, 446/446 | | N10 | detach `artifacts.frames.ppm` from the `WriteFileBytes` loop by closing it before the loop runs | 1 file, +1/-1 | rc 0 | **RED**, exit 1, 1 case failed, **527 assertions, 3 failed** | exit 0, 446/446 | +**N10 was RECONSTRUCTED, and it is named as such.** The fourth review's own +definition of it is not recorded in this tree or on the forge, so what runs under +that label here is the mutation the name describes: close `artifacts.frames.ppm` +before the `WriteFileBytes` loop instead of after it, which detaches the writer's +anchor from the writes while leaving the leaf, the count and the nesting intact. +The other nine are the reviews' own. + Every mutation was applied by exact-text replacement that refuses unless the pattern occurs exactly once, with its `git diff --stat` and its compile return code printed and the tree restored byte-for-byte and re-verified with @@ -1320,6 +1333,18 @@ never applies, reads exactly like a passing test. a transfer, an anchor that moved with its leaf, or a pair of anchor names that were exchanged. +**The four suites on that same head**, x86 CPU, Release, `-DVLLM_CPP_SERVER=OFF`: +`test_ltx2_video` exit 0, 96/96 cases, 3990/3990 assertions; `test_capi` exit 0, +65/65, 654/654; `test_ltx2_vae` exit 0, 43/43, 3125/3125; `test_ltx2_tiling` +exit 0, 10/10, 915/915. + +**One run of `test_ltx2_video` reported three cases THREW and it was not a +verdict.** The box filled to 100% mid-run and the three exceptions read +`No space left on device` and `short write .../frame_000000.ppm`. A full disk +presents as a claim about the code, which is a shape this repository has hit +before with checker refusals; the numbers above are a re-run with 91 GB free. +Check `df` before believing a red. + **No timing conclusion is drawn from this box.** Every threshold above is a within-run share. The leaf sums moved 0.658 s, 0.706 s and 0.779 s across three runs of one binary at one geometry while this was written, and earlier runs of From 1609e1d08984f218655f2d1e771bca83ead63109 Mon Sep 17 00:00:00 2001 From: Ettore Di Giacinto Date: Thu, 20 Aug 2026 08:43:55 +0000 Subject: [PATCH 08/11] record(LTX25-RESIDENCY-W0): the mutation table takes the second merge head, and N6's count moved between them MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `main` moved twice while this branch was open and the whole ten-mutation set was re-applied after each merge. The table now names `99703ce5a` and carries that head's numbers. The re-run was not a formality. N6 reports 5 failed assertions on this head against 3 on `8c66ffa16`, and M12 reports 3 against 5, because `4712dac40` narrows `act(gate)` to the input dtype in `src/vt/cpu/cpu_ops.cpp` — on this render's own path. Both stay RED, so the verdict is unchanged; the counts are not, and carrying the earlier table forward would have published numbers no command on this tree reproduces. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code] --- .agents/specs/ltx25-device-residency.md | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/.agents/specs/ltx25-device-residency.md b/.agents/specs/ltx25-device-residency.md index 4b576cf80..7b2adb7b2 100644 --- a/.agents/specs/ltx25-device-residency.md +++ b/.agents/specs/ltx25-device-residency.md @@ -1296,18 +1296,21 @@ step, which no anchor wraps — and its share depends on the geometry. Measured the measured value in both cases, which is the rule the other thresholds are set by. -**Re-run on the FINAL head**, the merge commit `8c66ffa16`, and not on the head each mutation was -first measured on: `origin/main` moved twice while this branch was open and one -of the incoming commits (`d0eff4f25`, W0-LIVE) edits this row's own driver, so a -mutation proof inherited from an earlier head is a claim about a tree that no -longer exists. +**Re-run on the FINAL head**, the merge commit `99703ce5a`, and not on the head +each mutation was first measured on. `origin/main` moved TWICE while this branch +was open: `d0eff4f25` (W0-LIVE) edits this row's own driver, and `4712dac40` +narrows `act(gate)` to the input dtype in `src/vt/cpu/cpu_ops.cpp`, which is on +this render's path. A mutation proof inherited from an earlier head is a claim +about a tree that no longer exists, and the whole set was re-applied after each +merge rather than carried forward — N6's count moved from 3 failed to 5 between +the two heads, which is what carrying it forward would have hidden. | # | Mutation | `git diff --stat` | compile | containment case | SUMS case | |---|---|---|---|---|---| | — | honest | (none) | rc 0 | exit 0, 1/1 cases, **527/527** | exit 0, 1/1, **446/446** | | **N1** | swap the `decode.audio.mel` and `decode.audio.vocoder` scope names — the **sibling-swap** mutation | 1 file, +2/-2 | rc 0 | **RED**, exit 1, 1 case failed, **527 assertions, 3 failed**, on the sibling order in each of the three per-render checks | exit 0, 446/446 | -| **N6** | move `decode.video.vae` off `AccumulateTemporalGroup` onto `buffer.Allocate` — the **anchor beside the work** mutation | 1 file, +7/-7 | rc 0 | **RED**, exit 1, 1 case failed, **527 assertions, 3 failed**, on the vae coverage floor | exit 0, 446/446 | -| M12 | re-anchored: `denoise.step` for step 0 only, `denoise` closed there, `phase.finish` over steps 1-7 | 1 file, +8/-1 | rc 0 | **RED**, exit 1, 1 case failed, **462 assertions, 5 failed** | exit 0, 374/374 | +| **N6** | move `decode.video.vae` off `AccumulateTemporalGroup` onto `buffer.Allocate` — the **anchor beside the work** mutation | 1 file, +7/-7 | rc 0 | **RED**, exit 1, 1 case failed, **527 assertions, 5 failed**, on the vae coverage floor | exit 0, 446/446 | +| M12 | re-anchored: `denoise.step` for step 0 only, `denoise` closed there, `phase.finish` over steps 1-7 | 1 file, +8/-1 | rc 0 | **RED**, exit 1, 1 case failed, **462 assertions, 3 failed** | exit 0, 374/374 | | M11 | re-anchored: writes stay inside `decode.video`, `artifacts.frames` emitted empty | 1 file, +3/-4 | rc 0 | **RED**, exit 1, 1 case failed, **527 assertions, 4 failed** | exit 0, 446/446 | | M11b | re-anchored: both `Close`s below the write, writer still around the loop | 1 file, +2/-2 | rc 0 | **RED**, exit 1, 1 case failed, **119 assertions, 1 failed** | exit 0, 446/446 | | M13 | re-anchored: swap the `load.dit` and `load.prompt_embeds` scope names | 1 file, +2/-2 | rc 0 | **RED**, exit 1, 1 case failed, **527 assertions, 2 failed**, on the load order | exit 0, 446/446 | From 8b82ad5caab5480142d1d5d06fafe6e2aac45e24 Mon Sep 17 00:00:00 2001 From: Ettore Di Giacinto Date: Thu, 20 Aug 2026 10:36:15 +0000 Subject: [PATCH 09/11] test(LTX25-RESIDENCY-W0): the SIBLING BOUNDARY moves 100% of a leaf onto the wrong name, and min_coverage sums MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `min_coverage` is checked on the SUM of a leaf's parts, so one part covering everything satisfies it and its siblings may measure nothing. A fifth fresh review reached that with two lines and no name change: leave `decode.audio.mel` open across the vocoder call and let `decode.audio.vocoder` open and close EMPTY beside it. Identical semantics, identical call order, only the instrument moves. The gate returned exit 0, 1/1, 527/527 — byte-identical to the honest tree — while the vocoder, the model #1010 asks for BY NAME, reported six microseconds of a half-second leaf. That is the same 100% attribution transfer the sibling NAME-SWAP mutation N1 produces, and N1 is red. Every other assertion survived it, and each for its own reason. (1b) passes because the mel still OPENS first, so it proves an order and not an identity. The counts are {1, 1} either way. Containment, `nested` and non-overlap all hold. This is round four's lesson again: assertion (6) was applied to the anchor that failed rather than to the property that failed. `Carrying::part_min_coverage` is the repair — a per-part floor against the leaf, required rather than defaulted and placed before `render` so a future leaf cannot acquire a second part without answering the question. `decode.audio` is the only multi-part leaf today, which is why the hole existed on exactly one pair and why (1b) is vacuous everywhere else. Only the vocoder gets a number, and the mel's absence is measured rather than assumed. Honest the vocoder holds 97.85%, 91.57%, 90.6% and 89.7% of its leaf here and 97.0% and 99.99% on the 21B artifact; under the mutation it holds 0.00045% and 0.0013%. Four to five orders, which is the separation the 0.50 `decode.video.vae` floor already exploits, so this floor is 0.50 too rather than tightened toward a share this contended box destroys. The MIRROR — mel emptied, the vocoder covering both calls — stays GREEN and is recorded as open in `### Owed out of W0` with the reason. Honest the mel holds 0.0004% to 2.9% on the 21B artifact; under the mirror it holds 0.0032%. The honest render's share is SMALLER than the mutation's, so the distributions overlap and any floor that reddens the mirror also reddens an honest render this row has already produced. Closing it needs a scope inside `Ltx2AudioDecoderForward`, in the shape `decode.video.vae` ships, and W1 is told in as many words that it is ranking a pair and not two models until then. Two stated reasons also claimed more than their numbers do. The vae denominator was justified by "without a second threshold for the reopen", but the reopen is itself a `decode.video.chunk` record and sits inside both candidate denominators: measured here the leaf is 0.00105035 s against 0.00103919 s of chunk, 1.06%, and the leaf is the marginally stricter of the two. And (1b) compares FIRST records, which is the whole order only because these parts do not interleave; a leaf with two repeating parts would satisfy it while running `A, B, B, A`. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code] --- .agents/specs/ltx25-device-residency.md | 50 +++++++- tests/vllm/multimodal/test_ltx2_video.cpp | 136 +++++++++++++++++++++- 2 files changed, 180 insertions(+), 6 deletions(-) diff --git a/.agents/specs/ltx25-device-residency.md b/.agents/specs/ltx25-device-residency.md index 7b2adb7b2..e16cf1a86 100644 --- a/.agents/specs/ltx25-device-residency.md +++ b/.agents/specs/ltx25-device-residency.md @@ -1412,13 +1412,61 @@ which is where this spec's `### Decisions taken here` already said they would be |---|---| | It runs once per unit of work the RENDER counted | **PROVEN**, at N = 8 for `denoise.step` and at N = 1 and N = 2 for every decode-side count, over two renders in one table. The counters come from `Ltx2ConditioningTrace` and the chunk count is re-derived from `Ltx2GroupTilesByTemporalSlice` | | Sibling anchors under one leaf are in the driver's own order | **PROVEN** for the one sibling pair that exists (`decode.audio.mel` before `decode.audio.vocoder`), and it is an ORDER, not an identity | + | A sibling anchor does not carry ITS SIBLING'S seconds | **HALF PROVEN**, and the half that is open is named below. A per-part floor holds `decode.audio.vocoder` at 50% of its leaf; nothing holds `decode.audio.mel`, for a measured reason | | Each anchor covers the CALL it is named after | **NOT PROVEN, for any of the six.** `denoise.step`, `decode.video.chunk`, `decode.audio.mel`, `decode.audio.vocoder` and `artifacts.frames.ppm` are all opened by a statement ADJACENT to the call. `decode.video.vae` is the only one whose ends are both production events, and what holds its placement is a coverage floor against `decode.video.chunk` — a ratio between two anchors, which is weaker than a count and stronger than the nothing that held it at `165db635c` | - The third row is the same debt `load.dit` carries and it is not smaller for + The last row is the same debt `load.dit` carries and it is not smaller for being on the render side. Closing it needs a scope inside the callee — `Ltx2AudioDecoderForward`, `Ltx2VocoderWithBweForward`, `Ltx2LoadDitFromSafetensors` — which is where the placement stops being a statement about where somebody wrote a line. + +* **THE SIBLING BOUNDARY MOVES 100% OF A LEAF'S SECONDS BETWEEN TWO NAMES, AND + ONE OF ITS TWO DIRECTIONS IS STILL OPEN.** This is stated separately from the + row above because it is a different failure: that row is a PRECISION debt — an + anchor opened by an adjacent statement measures a few microseconds more than + its call. This is a whole-attribution TRANSFER, and it is reached without + touching a scope's name, its count, its containment or its order. + + The shape, which a fifth fresh review named as mutation R1b: leave + `decode.audio.mel` open across the vocoder call and let + `decode.audio.vocoder` open and close EMPTY beside it + (`ltx2_video.cpp:4674-4682`, two lines, identical semantics and identical call + order). Assertion (1b) passes because the mel still OPENS first, so it proves + an order and not an identity; the counts are `{1, 1}` either way; containment, + `nested` and non-overlap all hold; and `min_coverage` is checked on the SUM of + the parts, so one part covering everything satisfies it. At `1609e1d08` the + gate returned exit 0, 1/1 and 527/527 assertions — byte-identical to the + honest tree — while the vocoder, the model + [#1010](https://github.com/mudler/vllm.cpp/issues/1010) asks for **by name**, + reported six microseconds of a half-second leaf. + + **The vocoder direction is now closed** by a per-part floor of 0.50 against the + `decode.audio` leaf (assertion (2b), `Carrying::part_min_coverage`). Honest the + vocoder measures 97.85%, 91.57%, 90.6% and 89.7% here and 97.0% and 99.99% on + the 21B artifact; under R1b it measures 0.00045% and 0.0013%. R1b now reds: + **exit 1, 1 case failed, 554 assertions, 3 failed.** + + **The MIRROR direction is NOT closed** — mel emptied, the vocoder scope + covering both calls — and no floor is available, which is a measurement and + not an omission. Honest, `decode.audio.mel` holds 2.13%, 8.43%, 9.4% and 10.3% + on this fixture and **0.0004% to 2.9%** on the 21B artifact. Under the mirror + it holds **0.0032% and 0.0020%**. The honest 21B render's 0.0004% is *smaller* + than the mirror's 0.0032%, so the two distributions overlap and any threshold + that reddens the mirror also reddens an honest render this row has already + produced. The mirror is measured GREEN: exit 0, 1/1, 554/554. + + **What W1 inherits, in one sentence, so that it does not inherit it by + silence:** `decode.audio.mel` may be carrying the vocoder's seconds and no gate + in this repository would say so. The cost is bounded in one direction — the + mirror overstates the vocoder by at most the mel's own share, and the vocoder + is the name #1010 already asks for — and unbounded in the other, which is + exactly why the vocoder got the floor first. Closing the mirror needs the same + thing the row above needs and nothing weaker: a scope INSIDE + `Ltx2AudioDecoderForward`, so that both of the mel anchor's ends are production + events, in the shape `decode.video.vae` already ships. **Any W1 ranking that + separates the audio VAE decode from the vocoder must land that scope first, or + state that it is ranking a pair and not two models.** * **A LEAF MAY STILL GROW OVER TIME NOBODY NAMED, and the room is measurable.** The `nested` invariant sees a leaf swallow a NEIGHBOUR, because the neighbour turns `nested` and leaves the timeline. Nothing sees a leaf swallow diff --git a/tests/vllm/multimodal/test_ltx2_video.cpp b/tests/vllm/multimodal/test_ltx2_video.cpp index 7be6159a2..3152bd6a1 100644 --- a/tests/vllm/multimodal/test_ltx2_video.cpp +++ b/tests/vllm/multimodal/test_ltx2_video.cpp @@ -3108,6 +3108,30 @@ struct Carrying { // note above this case. std::vector part_counts; std::string count_source; // where those numbers came from, for the message + // EACH PART'S OWN FLOOR, as a share of the LEAF's seconds. One entry per name + // in `parts`, and 0.0 means "no floor, and the row's spec says which direction + // that leaves open". + // + // WHY IT IS PER PART AND NOT ONE NUMBER. `min_coverage` above is checked on + // the SUM of the parts, so ONE part covering everything satisfies it and the + // others may measure nothing. That is not a hypothetical: mutation R1b leaves + // `decode.audio.mel` open across the vocoder call and lets + // `decode.audio.vocoder` open and close EMPTY beside it — identical semantics, + // identical call order, two lines — and it moves ~100% of `decode.audio`'s + // decomposed seconds onto the wrong name, which is the same end state the + // NAME-SWAP mutation N1 produces. Every other assertion here survives it: + // (1b) passes because mel still OPENS first, so it proves an ORDER and not an + // IDENTITY; the counts are {1, 1} either way; containment, `nested` and + // non-overlap all hold. This field is what makes the two names measurable + // SEPARATELY. + // + // IT IS REQUIRED RATHER THAN DEFAULTED, and deliberately sits before `render` + // so that a future leaf cannot acquire a second part without answering the + // question. `decode.audio{mel, vocoder}` is the ONLY multi-part leaf today — + // `denoise`, `decode.video` and `artifacts.frames` have one part each, which + // is why (1b) and this assertion are both vacuous for them and why the hole + // above existed on exactly one pair. + std::vector part_min_coverage; // WHICH RENDER. The table is a process timeline and every count above is a // per-render quantity; see `RecordsNamed`. int64_t render = kAnyRender; @@ -3162,9 +3186,14 @@ void CheckCarryingPhase(const nlohmann::json& table, const Carrying& c) { // ...and the FIRST start of each part name, in the order `parts` lists them, // which is assertion (1b) below. std::vector part_first; + // ...and each part's OWN seconds, kept separate from `subs` because `subs` + // collapses every part into one pool and assertion (2b) is precisely the + // question that pool cannot answer. + std::vector part_seconds; for (const std::string& part : c.parts) { INFO("sub-scope = " << part); const std::vector found = RecordsNamed(table, part, c.render); + double seconds = 0.0; REQUIRE_MESSAGE(!found.empty(), "the table names no '" << part << "' record. It is the anchor that ties the '" << c.leaf << "' name to the work beneath it, and without it this phase is " @@ -3190,7 +3219,9 @@ void CheckCarryingPhase(const nlohmann::json& table, const Carrying& c) { "different places, which is the whole defect this " "case exists for"); subs.push_back({start, end}); + seconds += end - start; } + part_seconds.push_back(seconds); double first = found.front()["start_seconds"].get(); for (const nlohmann::json& r : found) { first = std::min(first, r["start_seconds"].get()); @@ -3221,6 +3252,15 @@ void CheckCarryingPhase(const nlohmann::json& table, const Carrying& c) { // under `### Owed out of W0`. Compared on the FIRST record of each name, so a // part that legitimately repeats (`decode.video.chunk`) is unaffected, and the // assertion is vacuous for a leaf with a single part name. + // + // AND COMPARING FIRST RECORDS IS THE WHOLE ORDER ONLY BECAUSE THESE PARTS DO + // NOT INTERLEAVE. `decode.audio.mel` and `decode.audio.vocoder` emit one + // record each, so `first(A) <= first(B)` is exact. A future leaf with two + // REPEATING parts would satisfy this assertion while running `A, B, B, A, B, + // A` — the first pair is in order and every pair after it is not. That leaf + // owes a per-record pairing, and this one does not have to pay for it: the + // (0) count is what would make the difference visible, since `{1, 1}` is the + // premise the line above rests on. for (size_t i = 1; i < part_first.size(); ++i) { INFO("sub-scope pair = " << c.parts[i - 1] << " then " << c.parts[i]); CHECK_MESSAGE(part_first[i] >= part_first[i - 1] - 1e-9, @@ -3254,6 +3294,66 @@ void CheckCarryingPhase(const nlohmann::json& table, const Carrying& c) { << "% this phase's anchor is expected to reach. The rest is a phase nobody " "named, wearing this one's label"); + // (2b) AND EACH PART'S OWN SHARE, because (2) is checked on the SUM. + // + // WHAT IT CLOSES, and it is the fifth transfer shape this case has been shown. + // Assertion (2) permits ONE part to cover the whole leaf while its siblings + // measure nothing, and the SIBLING BOUNDARY is enough to do it: leave + // `decode.audio.mel` open across the vocoder call and open `decode.audio. + // vocoder` empty after it (mutation R1b, two lines, identical semantics and + // identical call order) and the vocoder — the model #1010 asks for BY NAME — + // reports 16 microseconds of a half-second leaf, with (0), (1b), (1), (2), + // (3), the `nested` invariant and the non-overlap check all green. It is the + // same 100% attribution transfer N1 achieves by swapping the two literals, + // reached without touching either literal. + // + // ONLY THE VOCODER CARRIES A FLOOR, and the mel's 0.0 is a MEASURED debt + // rather than an oversight. Honest, over the two renders of this case on two + // runs of this box, `decode.audio.vocoder` holds 97.85%, 91.57%, 90.6% and + // 89.7% of its leaf; on the 21B artifact this row shipped it holds 97.0% and + // 99.99%. Under R1b it holds 0.00045% and 0.0013%. Between four and five + // orders of separation, which is the property the 0.50 `decode.video.vae` + // floor below exploits, and why this number is 0.50 rather than tightened + // toward a measured share this box destroys. + // + // THE MEL CANNOT HAVE ONE, AND THE REASON IS THAT THE TWO DISTRIBUTIONS + // OVERLAP. Honest, `decode.audio.mel` holds 2.13% and 8.43% here (9.4% and + // 10.3% on the other run) and 0.0004% to 2.9% on the artifact, because the + // audio VAE decode's cost RELATIVE to the vocoder is a property of the + // geometry. Under the MIRROR of R1b — mel emptied, the vocoder scope covering + // both calls — it holds 0.0032% and 0.0020%. The honest 21B render's 0.0004% + // is SMALLER than the mirror's 0.0032%, so no threshold separates them: any + // floor that reddens the mirror also reddens an honest render this row has + // already produced. The mirror is therefore measured, GREEN, and recorded as + // open in `### Owed out of W0` rather than closed by a number that cannot be + // justified. + // + // The two directions are not equally expensive, and that is why one is closed + // and one is disclosed. R1b moves the VOCODER's 90-99.99% onto the mel, which + // is the 100% attribution transfer that would have made W1 rank the wrong + // model. The mirror moves the MEL's 0.0004-10% onto the vocoder, so it + // overstates the name #1010 already asks for by at most the smaller share. + // + // AND IT IS A FLOOR, NOT A BAND. A partial transfer — an anchor covering half + // its call — passes 0.50 and is not detected here. Closing that needs a scope + // INSIDE the callee, which is the same debt the third row of the anchor table + // in `### Owed out of W0` carries for all six anchors. + REQUIRE(c.part_min_coverage.size() == c.parts.size()); + REQUIRE(part_seconds.size() == c.parts.size()); + for (size_t i = 0; i < c.parts.size(); ++i) { + INFO("sub-scope = " << c.parts[i]); + MESSAGE(" " << c.parts[i] << " = " << part_seconds[i] << "s, " + << (100.0 * part_seconds[i] / leaf_seconds) << "% of " << c.leaf); + if (c.part_min_coverage[i] <= 0.0) continue; + CHECK_MESSAGE(part_seconds[i] >= c.part_min_coverage[i] * leaf_seconds, + "'" << c.parts[i] << "' holds " << part_seconds[i] << "s of the " + << leaf_seconds << "s its '" << c.leaf << "' leaf spent, under the " + << (100.0 * c.part_min_coverage[i]) + << "% this name is expected to reach. A SIBLING scope that opened around " + "this one's call carries its seconds under the sibling's name, and " + "every other assertion in this case survives that"); + } + // (3) EXCLUSIVITY. Nothing else may run in the window the anchor spans. This // is the assertion that sees a TRANSFER directly rather than through a sum: // M7 opens `phase.finish` in the middle of the denoise, and the denoise's own @@ -3473,6 +3573,10 @@ void CheckRenderPhases(const nlohmann::json& table, {}, {trace.dit_evaluations}, "Ltx2ConditioningTrace::dit_evaluations", + // ONE part, so (2) already IS the per-part assertion and a second copy of + // it would only be noise. `denoise.step`'s own placement debt is the + // third row of the anchor table in `### Owed out of W0`. + {0.0}, render}, {"decode.video", {"decode.video.chunk"}, @@ -3480,6 +3584,10 @@ void CheckRenderPhases(const nlohmann::json& table, {"artifacts.frames"}, {trace.video_decode_chunks + 1}, "Ltx2ConditioningTrace::video_decode_chunks + 1, the reopen after the last chunk", + // One part; and this leaf's real per-part floor is the `decode.video.vae` + // block below, which is a ratio between two ANCHORS rather than between an + // anchor and its leaf. + {0.0}, render}, {"decode.audio", {"decode.audio.mel", "decode.audio.vocoder"}, @@ -3487,6 +3595,11 @@ void CheckRenderPhases(const nlohmann::json& table, {}, {1, 1}, "the two calls #1010 names, once each per render", + // THE ONLY MULTI-PART LEAF, and therefore the only place (2b) is not + // vacuous. The vocoder carries 0.50 and the mel carries none; the note on + // `part_min_coverage` and on (2b) argues both numbers, and + // `### Owed out of W0` records what the mel's 0.0 leaves open. + {0.0, 0.50}, render}, // AND THE WRITER, which shipped with no anchor at all and is the leaf the // `decode.video` number is protected by. `artifacts.frames.ppm` wraps the @@ -3515,6 +3628,8 @@ void CheckRenderPhases(const nlohmann::json& table, {"decode.video"}, {trace.video_decode_chunks}, "Ltx2ConditioningTrace::video_decode_chunks, one write callback per chunk", + // One part, so (2) already is it. + {0.0}, render}, }; for (const Carrying& c : carrying) CheckCarryingPhase(table, c); @@ -3543,11 +3658,22 @@ void CheckRenderPhases(const nlohmann::json& table, // green. The claim that this leaf "finally has a sub-scope whose ends are both // production events" was true of the source and unheld by the gate. // - // THE DENOMINATOR IS `decode.video.chunk`, not the leaf: the leaf's last chunk - // record is the empty window between the final chunk and the end of the - // decode, and both are inside the same leaf, so the chunk windows are the - // tightest thing the vae can be measured against without a second threshold - // for the reopen. Measured at 91.1%, 93.6%, 97.4%, 98.2%, 98.5% and 98.6% of + // THE DENOMINATOR IS `decode.video.chunk`, not the leaf, AND THE REOPEN IS NOT + // THE REASON. This note used to say the chunk windows are the tightest + // denominator available "without a second threshold for the reopen", which + // claims more than the numbers do. The reopen — the empty window between the + // final chunk and the end of the decode — is itself a `decode.video.chunk` + // record, so it sits inside BOTH candidate denominators and separates neither. + // Measured here, render 1: the leaf is 0.00105035 s against 0.00103919 s of + // chunk, a 1.06% difference that is two scope boundaries; render 2: + // 0.00514612 s against 0.00512619 s, 0.39%. The two denominators are within a + // percent of each other and the LEAF is the marginally stricter of the two, so + // nothing is loosened by this choice. + // + // The chunk is used because it is the same window the containment assertion + // below encloses each vae record IN, which keeps this ratio between two + // quantities at one granularity rather than comparing a per-group anchor + // against a whole-decode leaf. Measured at 91.1%, 93.6%, 97.4%, 98.2%, 98.5% and 98.6% of // the chunk seconds across six renders here — the low end is the nine-frame // render, whose single chunk is about a millisecond. The floor is 0.50 // because what is NOT covered is the per-group From 9e40c68445e49b310392ee34fc229509c9b53674 Mon Sep 17 00:00:00 2001 From: Ettore Di Giacinto Date: Thu, 20 Aug 2026 12:32:40 +0000 Subject: [PATCH 10/11] record(LTX25-RESIDENCY-W0): twelve mutations on the fourth merge head, and two RECONSTRUCTIONS that reddened for the wrong reason MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The set is re-applied on `17eba8ce2` rather than carried forward, and the counts moved again while every verdict held. N6 reports 3 failed assertions here against 5 on `99703ce5a`. N1 reports 6 against 3, and that one is this change's own doing: the per-part floor catches the name swap as well as the boundary shift, because under N1 the vocoder name lands on the mel's work and reads 2.76% and 7.68% of its leaf. Every share in the code and in the spec is now this head's. Honest the vocoder holds 99.46% and 94.10%; under R1b 0.00091% and 0.0035%; under the mirror the mel holds 0.00063% and 0.0051% against an honest 0.53% and 5.90% here and 0.0004% on the 21B artifact. That last pair is the whole argument for leaving the mirror open: the honest artifact share is SMALLER than the mutation's, so no threshold separates them. Seven of the twelve are reconstructions and the record now says which. Only R1b, its mirror, N1, N6 and M13 run from a definition recorded in this tree; the earlier reviews' own M12, M11, M11b, M7, M4, M10 and N10 are not in the tree or on the forge, so each is rebuilt from the row that describes it. TWO OF THOSE REBUILDS WERE WRONG, and a bare RED is exactly what would have hidden it. A first M4 that never re-closed `decode.video` reddened the SUMS case, which the recorded M4 does not. A first M12 closed `denoise.step` early instead of suppressing it — closing still EMITS the record — which left M12 and M7 byte-identical in their verdicts: 556 assertions, 26 failed, on the same five failure texts. Both were repaired until the failure SHAPE matched the recorded one, M4 aborting on a `REQUIRE_FALSE(nested)` and M12 failing first on `'denoise.step' was emitted 1 time(s)` against `dit_evaluations` = 8. A reconstruction that merely reds is not evidence that the recorded mutation reds. The vae denominator note also takes this head's numbers: leaf 0.191059 s against 0.191044 s of chunk, 0.0079%, where an earlier run of the same binary at the same geometry read 1.06%. The boundary cost is fixed and the leaf is not, which is the box rather than the structure, and it is why no duration here is compared with any other. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code] --- .agents/specs/ltx25-device-residency.md | 79 +++++++++++++++-------- tests/vllm/multimodal/test_ltx2_video.cpp | 49 +++++++------- 2 files changed, 79 insertions(+), 49 deletions(-) diff --git a/.agents/specs/ltx25-device-residency.md b/.agents/specs/ltx25-device-residency.md index d4193276d..b134e049f 100644 --- a/.agents/specs/ltx25-device-residency.md +++ b/.agents/specs/ltx25-device-residency.md @@ -1307,24 +1307,45 @@ the two heads, which is what carrying it forward would have hidden. | # | Mutation | `git diff --stat` | compile | containment case | SUMS case | |---|---|---|---|---|---| -| — | honest | (none) | rc 0 | exit 0, 1/1 cases, **527/527** | exit 0, 1/1, **446/446** | -| **N1** | swap the `decode.audio.mel` and `decode.audio.vocoder` scope names — the **sibling-swap** mutation | 1 file, +2/-2 | rc 0 | **RED**, exit 1, 1 case failed, **527 assertions, 3 failed**, on the sibling order in each of the three per-render checks | exit 0, 446/446 | -| **N6** | move `decode.video.vae` off `AccumulateTemporalGroup` onto `buffer.Allocate` — the **anchor beside the work** mutation | 1 file, +7/-7 | rc 0 | **RED**, exit 1, 1 case failed, **527 assertions, 5 failed**, on the vae coverage floor | exit 0, 446/446 | -| M12 | re-anchored: `denoise.step` for step 0 only, `denoise` closed there, `phase.finish` over steps 1-7 | 1 file, +8/-1 | rc 0 | **RED**, exit 1, 1 case failed, **462 assertions, 3 failed** | exit 0, 374/374 | -| M11 | re-anchored: writes stay inside `decode.video`, `artifacts.frames` emitted empty | 1 file, +3/-4 | rc 0 | **RED**, exit 1, 1 case failed, **527 assertions, 4 failed** | exit 0, 446/446 | -| M11b | re-anchored: both `Close`s below the write, writer still around the loop | 1 file, +2/-2 | rc 0 | **RED**, exit 1, 1 case failed, **119 assertions, 1 failed** | exit 0, 446/446 | -| M13 | re-anchored: swap the `load.dit` and `load.prompt_embeds` scope names | 1 file, +2/-2 | rc 0 | **RED**, exit 1, 1 case failed, **527 assertions, 2 failed**, on the load order | exit 0, 446/446 | -| M7 | re-anchored: close `denoise` after the first sampler step and open `phase.finish` there | 1 file, +6/-0 | rc 0 | **RED**, exit 1, 1 case failed, **539 assertions, 24 failed** | exit 0, 458/458 | -| M4 | re-anchored: `decode.video` left open across the audio decode | 1 file, +2/-1 | rc 0 | **RED**, exit 1, 1 case failed, **88 assertions, 2 failed** (a `REQUIRE_FALSE` aborts the case early) | exit 0, 446/446 | -| M10 | re-anchored: after a chunk, re-label the decode as the writer | 1 file, +1/-1 | rc 0 | **RED**, exit 1, 1 case failed, **529 assertions, 5 failed** | exit 0, 446/446 | -| N10 | detach `artifacts.frames.ppm` from the `WriteFileBytes` loop by closing it before the loop runs | 1 file, +1/-1 | rc 0 | **RED**, exit 1, 1 case failed, **527 assertions, 3 failed** | exit 0, 446/446 | - -**N10 was RECONSTRUCTED, and it is named as such.** The fourth review's own -definition of it is not recorded in this tree or on the forge, so what runs under -that label here is the mutation the name describes: close `artifacts.frames.ppm` -before the `WriteFileBytes` loop instead of after it, which detaches the writer's -anchor from the writes while leaving the leaf, the count and the nesting intact. -The other nine are the reviews' own. +| — | honest | (none) | rc 0 | exit 0, 1/1 cases, **554/554** | exit 0, 1/1, **446/446** | +| **R1b** | leave `decode.audio.mel` open across the vocoder call and open `decode.audio.vocoder` EMPTY beside it — the **sibling-boundary** mutation, and the fifth review's finding | 1 file, +2/-2 | rc 0 | **RED**, exit 1, 1 case failed, **554 assertions, 3 failed**, on the per-part floor in each of the three per-render checks | exit 0, 446/446 | +| **R1b mirror** | the same shift the other way: `decode.audio.mel` emptied, `decode.audio.vocoder` covering both calls | 1 file, +2/-2 | rc 0 | **GREEN**, exit 0, 1/1, 554/554 — **OPEN, and recorded above with the measurement that says no floor separates it** | exit 0, 446/446 | +| **N1** | swap the `decode.audio.mel` and `decode.audio.vocoder` scope names — the **sibling-swap** mutation | 1 file, +2/-2 | rc 0 | **RED**, exit 1, 1 case failed, **554 assertions, 6 failed** — three on the sibling order and three on the per-part floor, which is new | exit 0, 446/446 | +| **N6** | move `decode.video.vae` off `AccumulateTemporalGroup` onto `buffer.Allocate` — the **anchor beside the work** mutation | 1 file, +4/-2 | rc 0 | **RED**, exit 1, 1 case failed, **554 assertions, 3 failed**, on the vae coverage floor | exit 0, 446/446 | +| M12 | re-anchored: `denoise.step` emitted for the first evaluation only, `denoise` closed at step 0, `phase.finish` over the rest | 1 file, +8/-1 | rc 0 | **RED**, exit 1, 1 case failed, **479 assertions, 5 failed**, first on `'denoise.step' was emitted 1 time(s)` against `dit_evaluations` = 8 | exit 0, 374/374 | +| M11 | re-anchored: writes stay inside `decode.video`, `artifacts.frames` emitted empty | 1 file, +3/-3 | rc 0 | **RED**, exit 1, 1 case failed, **126 assertions, 1 failed** | exit 0, 446/446 | +| M11b | re-anchored: both `Close`s below the write, writer still around the loop | 1 file, +2/-2 | rc 0 | **RED**, exit 1, 1 case failed, **126 assertions, 1 failed** | exit 0, 446/446 | +| M13 | re-anchored: swap the `load.dit` and `load.prompt_embeds` scope names | 1 file, +2/-2 | rc 0 | **RED**, exit 1, 1 case failed, **554 assertions, 2 failed**, on the load order | exit 0, 446/446 | +| M7 | re-anchored: close `denoise` after the first sampler step and open `phase.finish` there | 1 file, +5/-0 | rc 0 | **RED**, exit 1, 1 case failed, **556 assertions, 26 failed** | exit 0, 458/458 | +| M4 | re-anchored: `decode.video` left open across the audio decode | 1 file, +1/-1 | rc 0 | **RED**, exit 1, 1 case failed, **92 assertions, 2 failed** (a `REQUIRE_FALSE` aborts the case early) | exit 0, 446/446 | +| M10 | re-anchored: after a chunk, re-label the decode as the writer | 1 file, +1/-1 | rc 0 | **RED**, exit 1, 1 case failed, **556 assertions, 4 failed** | exit 0, 446/446 | +| N10 | detach `artifacts.frames.ppm` from the `WriteFileBytes` loop by closing it before the loop runs | 1 file, +1/-1 | rc 0 | **RED**, exit 1, 1 case failed, **554 assertions, 3 failed** | exit 0, 446/446 | + +**Twelve mutations on `17eba8ce2`, the head that lands**, re-applied after the +FOURTH merge rather than carried forward. The counts moved again and the +verdicts did not: N6 reports 3 failed assertions here against 5 on `99703ce5a`, +and N1 reports 6 against 3 — the last because the per-part floor this change adds +catches the name swap as well as the boundary shift. + +**SEVEN OF THE TWELVE ARE RECONSTRUCTIONS, and this is the disclosure.** Only +R1b, its mirror, N1, N6 and M13 run from a definition recorded in this tree. The +earlier reviews' own definitions of M12, M11, M11b, M7, M4, M10 and N10 are not +in this tree or on the forge, so what runs under those labels is the mutation +each NAME describes, rebuilt from the row beside it in this table — N10, for +instance, closes `artifacts.frames.ppm` before the `WriteFileBytes` loop instead +of after it, which detaches the writer's anchor from the writes while leaving the +leaf, the count and the nesting intact. + +**Two of the rebuilds were wrong on the first attempt, and a bare RED is what +would have hidden it.** A first M4 that never re-closed `decode.video` reddened +the SUMS case, which the recorded M4 does not; a first M12 closed `denoise.step` +early instead of SUPPRESSING it, which still emits the record, and that made M12 +indistinguishable from M7 — both 556 assertions with 26 failed, on the same five +failure texts. Both were repaired until the failure SHAPE matched the recorded +one: M4 aborting on a `REQUIRE_FALSE(nested)` after a coverage failure, M12 +failing first on `'denoise.step' was emitted 1 time(s)` against +`dit_evaluations` = 8. A reconstruction that merely reds is not evidence that the +recorded mutation reds, and two of seven here proved it. Every mutation was applied by exact-text replacement that refuses unless the pattern occurs exactly once, with its `git diff --stat` and its compile return @@ -1443,18 +1464,22 @@ which is where this spec's `### Decisions taken here` already said they would be **The vocoder direction is now closed** by a per-part floor of 0.50 against the `decode.audio` leaf (assertion (2b), `Carrying::part_min_coverage`). Honest the - vocoder measures 97.85%, 91.57%, 90.6% and 89.7% here and 97.0% and 99.99% on - the 21B artifact; under R1b it measures 0.00045% and 0.0013%. R1b now reds: - **exit 1, 1 case failed, 554 assertions, 3 failed.** + vocoder measures **99.46% and 94.10%** on the head this lands at (97.85%, + 91.57%, 90.6% and 89.7% on earlier runs) and 97.0% and 99.99% on the 21B + artifact; under R1b it measures **0.00091% and 0.0035%**. R1b now reds: + **exit 1, 1 case failed, 554 assertions, 3 failed.** The same floor also + catches the NAME SWAP, which is why N1 moved from 3 failed assertions to 6: + under N1 the vocoder name sits on the mel's work and reads 2.76% and 7.68%. **The MIRROR direction is NOT closed** — mel emptied, the vocoder scope covering both calls — and no floor is available, which is a measurement and - not an omission. Honest, `decode.audio.mel` holds 2.13%, 8.43%, 9.4% and 10.3% - on this fixture and **0.0004% to 2.9%** on the 21B artifact. Under the mirror - it holds **0.0032% and 0.0020%**. The honest 21B render's 0.0004% is *smaller* - than the mirror's 0.0032%, so the two distributions overlap and any threshold - that reddens the mirror also reddens an honest render this row has already - produced. The mirror is measured GREEN: exit 0, 1/1, 554/554. + not an omission. Honest, `decode.audio.mel` holds **0.53% and 5.90%** on this + head (2.13%, 8.43%, 9.4% and 10.3% on earlier runs) and **0.0004% to 2.9%** on + the 21B artifact. Under the mirror it holds **0.00063% and 0.0051%**. The + honest 21B render's 0.0004% is *smaller* than the mirror's 0.0051%, so the two + distributions overlap and any threshold that reddens the mirror also reddens an + honest render this row has already produced. The mirror is measured GREEN: + exit 0, 1/1, 554/554. **What W1 inherits, in one sentence, so that it does not inherit it by silence:** `decode.audio.mel` may be carrying the vocoder's seconds and no gate diff --git a/tests/vllm/multimodal/test_ltx2_video.cpp b/tests/vllm/multimodal/test_ltx2_video.cpp index 3152bd6a1..13c7f848f 100644 --- a/tests/vllm/multimodal/test_ltx2_video.cpp +++ b/tests/vllm/multimodal/test_ltx2_video.cpp @@ -3308,25 +3308,27 @@ void CheckCarryingPhase(const nlohmann::json& table, const Carrying& c) { // reached without touching either literal. // // ONLY THE VOCODER CARRIES A FLOOR, and the mel's 0.0 is a MEASURED debt - // rather than an oversight. Honest, over the two renders of this case on two - // runs of this box, `decode.audio.vocoder` holds 97.85%, 91.57%, 90.6% and - // 89.7% of its leaf; on the 21B artifact this row shipped it holds 97.0% and - // 99.99%. Under R1b it holds 0.00045% and 0.0013%. Between four and five - // orders of separation, which is the property the 0.50 `decode.video.vae` - // floor below exploits, and why this number is 0.50 rather than tightened - // toward a measured share this box destroys. + // rather than an oversight. Honest, over the two renders of this case, + // `decode.audio.vocoder` holds 99.46% and 94.10% of its leaf (97.85%, 91.57%, + // 90.6% and 89.7% on earlier runs of this box); on the 21B artifact this row + // shipped it holds 97.0% and 99.99%. Under R1b it holds 0.00091% and 0.0035%. + // Between four and five orders of separation, which is the property the 0.50 + // `decode.video.vae` floor below exploits, and why this number is 0.50 rather + // than tightened toward a measured share this box destroys. It also catches + // the NAME SWAP: under N1 the vocoder name lands on the mel's work and reads + // 2.76% and 7.68%, so that mutation now reds here as well as on (1b). // // THE MEL CANNOT HAVE ONE, AND THE REASON IS THAT THE TWO DISTRIBUTIONS - // OVERLAP. Honest, `decode.audio.mel` holds 2.13% and 8.43% here (9.4% and - // 10.3% on the other run) and 0.0004% to 2.9% on the artifact, because the - // audio VAE decode's cost RELATIVE to the vocoder is a property of the - // geometry. Under the MIRROR of R1b — mel emptied, the vocoder scope covering - // both calls — it holds 0.0032% and 0.0020%. The honest 21B render's 0.0004% - // is SMALLER than the mirror's 0.0032%, so no threshold separates them: any - // floor that reddens the mirror also reddens an honest render this row has - // already produced. The mirror is therefore measured, GREEN, and recorded as - // open in `### Owed out of W0` rather than closed by a number that cannot be - // justified. + // OVERLAP. Honest, `decode.audio.mel` holds 0.53% and 5.90% here (2.13%, + // 8.43%, 9.4% and 10.3% on earlier runs) and 0.0004% to 2.9% on the artifact, + // because the audio VAE decode's cost RELATIVE to the vocoder is a property of + // the geometry. Under the MIRROR of R1b — mel emptied, the vocoder scope + // covering both calls — it holds 0.00063% and 0.0051%. The honest 21B render's + // 0.0004% is SMALLER than the mirror's 0.0051%, so no threshold separates + // them: any floor that reddens the mirror also reddens an honest render this + // row has already produced. The mirror is therefore measured, GREEN, and + // recorded as open in `### Owed out of W0` rather than closed by a number that + // cannot be justified. // // The two directions are not equally expensive, and that is why one is closed // and one is disclosed. R1b moves the VOCODER's 90-99.99% onto the mel, which @@ -3664,11 +3666,14 @@ void CheckRenderPhases(const nlohmann::json& table, // claims more than the numbers do. The reopen — the empty window between the // final chunk and the end of the decode — is itself a `decode.video.chunk` // record, so it sits inside BOTH candidate denominators and separates neither. - // Measured here, render 1: the leaf is 0.00105035 s against 0.00103919 s of - // chunk, a 1.06% difference that is two scope boundaries; render 2: - // 0.00514612 s against 0.00512619 s, 0.39%. The two denominators are within a - // percent of each other and the LEAF is the marginally stricter of the two, so - // nothing is loosened by this choice. + // Measured on the head this lands at, render 1: the leaf is 0.191059 s against + // 0.191044 s of chunk, a 0.0079% difference that is two scope boundaries; + // render 2: 0.408779 s against 0.408758 s, 0.0051%. An earlier run of the same + // binary at the same geometry read 1.06% and 0.39%, because the boundary cost + // is fixed while the leaf is not — which is the box, not the structure. Either + // way the two denominators are within about a percent of each other and the + // LEAF is the marginally stricter of the two, so nothing is loosened by this + // choice. // // The chunk is used because it is the same window the containment assertion // below encloses each vae record IN, which keeps this ratio between two From 9d38ebda3d01784ebbb7818ef32ddc778560d0af Mon Sep 17 00:00:00 2001 From: Ettore Di Giacinto Date: Thu, 20 Aug 2026 12:54:20 +0000 Subject: [PATCH 11/11] record(LTX25-RESIDENCY-W0): the set reproduces byte-identically across the fifth merge, and main's own audit-live-rows red is filed MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The twelve mutations were re-run on `1160d04b5` and every count is identical to `17eba8ce2`. That is the evidence the fifth merge commit's argument was owed: it predicted the delta could not move the counts because `git diff --numstat` reads 87 insertions and zero deletions in `src/vt/cpu/cpu_ops.cpp`, and a prediction about a file is not a measurement of a render. Twelve unchanged counts are. `scripts/agent-preflight.sh` also reports `audit-live-rows` and `test_audit_live_rows` failing, and neither is in this row's known-red baseline. They are not this branch's. A detached worktree at `origin/main` `cae8ace0c` with nothing applied fails the same assertion with the same value — `stale ACTIVE rows remain: ['KERNEL-DFLASH2-GROUPED-CONV']` — so the red arrived with `028438e68`, whose implementation landed while its kernel-matrix row stayed `ACTIVE` under a live claim. Filed as #1469 and owned by `KERNEL-DFLASH2-GROUPED-CONV`, not fixed here: that row, the kernel matrix and the DFlash2 claim are outside this row's authority, and a lifecycle move owes `docs/STATUS.md`, `docs/BENCHMARKS.md` and the moved spec's `## Now`. Worth recording how it hides: `scripts/audit-live-rows.py` run on its own returns rc 0 while printing `1 abandoned ACTIVE`. The count is the gate and the exit status does not carry it, so the script looks green to anyone who checks it the obvious way. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code] --- .agents/issue-index.md | 1 + .agents/specs/ltx25-device-residency.md | 18 +++++++++++++----- 2 files changed, 14 insertions(+), 5 deletions(-) diff --git a/.agents/issue-index.md b/.agents/issue-index.md index ac0f1a55a..161856a04 100644 --- a/.agents/issue-index.md +++ b/.agents/issue-index.md @@ -489,3 +489,4 @@ rather than merged. `scripts/check-agent-record.py` gates both. | [#1447](https://github.com/mudler/vllm.cpp/issues/1447) | `ENG-EXPERT-STREAM` | **`docs/USAGE.md` said EVERY Qwen3.8 decode figure came from the W0e C ABI harness; the 66.7 s/token streaming-off row of 16 August 2026 came from `vllm-server`, the same binary the section tells the reader to run.** Introduced by #1211 and fixed in the same flow: the sentence is scoped to the W0e and W0f runs and the exception is named, in `docs/USAGE.md` and in the spec paragraph that mirrors it. Provenance read at the source, `.agents/specs/expert-streaming.md:837` (server entry point) and `:905`. | record | | [#1403](https://github.com/mudler/vllm.cpp/issues/1403) | `ENG-CUDAGRAPH-BREAK` | **`test_qwen3_5_decode_graph_seam` SIGSEGVs — a DUPLICATE of [#1394](https://github.com/mudler/vllm.cpp/issues/1394) that was already fixed when it was filed, so no code change was owed.** Filed against `96ed8346f`, which PREDATES the fix `7dec1d990` (PR #1393, merged 2026-08-20 00:49); `git merge-base --is-ancestor` puts the fix outside the reported tree. Re-measured at `b537a5344`, CPU-only Release x86_64: the test passes 5 runs of 5, `8 cases`/`8 passed`, `assertions: 138`, `Status: SUCCESS!`, exit 0. The cause was re-derived by single-variable mutation rather than inherited from #1394's report, and NAMED by a sanitizer rather than reasoned. Debug + `VLLM_CPP_SANITIZE=address,undefined` (NDEBUG OFF) with BOTH halves of `7dec1d990` reverted to their `96ed8346f` bytes: `AddressSanitizer: SEGV on unknown address 0x5045f5f84900`, `The signal is caused by a READ memory access`, at `src/vt/cpu/cpu_paged_attn.cpp:59` in `KvElem` reached from `:224`, on threadpool worker T1, inside case `W6: two spec shapes of EQUAL S and different q get two graphs` (`tests/vllm/models/test_qwen3_5_decode_graph_seam.cpp:800`). **It is NOT a use-after-free and there is no free site**, and `AddressSanitizer can not provide additional info` is the tell: the out-of-bounds block-table read is itself IN-BOUNDS for ASan because it lands inside the neighbouring pooled allocation, so this is not a `heap-buffer-overflow` either — the VALUE read becomes a wild block index that `KvElem` multiplies by the KV block stride and dereferences, which is why the fault address is unmapped rather than a redzone and why the crash is order-dependent instead of deterministic. Both halves are load-bearing: reverting ONLY the `SpecAttnMeta` fixture leaves the kernel bound to refuse by name at `src/vt/cpu/cpu_paged_attn.cpp:152` (exit 1, the case THROWS); reverting the bound as well restores the crash (Release exit 139, 3 runs of 3; ASan exit 1); restoring both gives exit 0, 8/8, zero sanitizer findings. The issue's `5f68e60df` pointer was right about the commit and wrong about the half — `git log -S'SpecAttnMeta'` and `git log -S'W6: two spec shapes of EQUAL S'` each return that commit ALONE, and both hits sit in the TEST fixture, not in the graph eligibility it widened. [#1405](https://github.com/mudler/vllm.cpp/issues/1405)'s truncation reproduced (`assertions: 135` crashed vs `138` complete) and is SHARPER under a threadpool fault: doctest printed the summary TWICE with different totals, `135` then `141`, because the main thread kept running after a worker died. That a red reached `main` unreported is [#1285](https://github.com/mudler/vllm.cpp/issues/1285)/[#1376](https://github.com/mudler/vllm.cpp/issues/1376), not this row | bug | | [#1458](https://github.com/mudler/vllm.cpp/issues/1458) | `VT-ACT-ROUND-POLARITY` | **`4712dac40` reds FOUR suites on `main` — `test_ltx2_text_encoder`, `test_muse_glimmer_text`, `test_muse_glimmer_text_fallback`, `test_minimax_music3_ar` — by exceeding bf16 error floors none of them had re-derived.** Found while gating [#1403](https://github.com/mudler/vllm.cpp/issues/1403) (PR #1457) and filed in flow, not that row's defect. Deterministic, not a load artifact: first seen under `-j4` and re-measured SERIALLY, 101 s across the four. `test_ltx2_text_encoder.cpp:2407` reads `CHECK( 0.1323 <= 0.109394 )` and `:2409` reads `CHECK( 0.0752773 <= 0.0573374 )`, both over by 20-30%, with the file at `26 passed / 1 failed`, `4118 assertions`, `Status: FAILURE!`. ATTRIBUTED BY MUTATION rather than inferred, both directions, in one build directory changing only `src/vt/cpu/cpu_ops.cpp`, compile rc 0 on every arm and `sha256sum` taken before and after: at `0adeb8b0e` four fail; reverted to `4712dac40^` four pass and `test_ltx2_text_encoder` is 27/27 `SUCCESS!`; restored to a re-matched `sha256` four fail again. They also passed at `b537a5344`, three commits earlier, in a full 567-test `ctest` run at 3-6 s each. `4712dac40` is +42/-3 in `src/vt/cpu/cpu_ops.cpp` plus a new `tests/vt/test_ops_activation.cpp`, and touches none of the four. NOT FIXED IN FLOW, deliberately: the question is a numerics decision, not a defect with an obvious repair — either the floors were calibrated against the rounding polarity that commit corrected and need re-deriving against the oracle, or the narrowing is wider than upstream's — and editing the four floors to pick the first branch is the scope-widening `AGENTS.md` prohibits | bug | +| [#1469](https://github.com/mudler/vllm.cpp/issues/1469) | `KERNEL-DFLASH2-GROUPED-CONV` | **`test_audit_live_rows` is RED on `main`.** Measured on a detached worktree at `origin/main` `cae8ace0c` with nothing applied: `test_shipped_record_has_no_abandoned_active_row` fails with `stale ACTIVE rows remain: ['KERNEL-DFLASH2-GROUPED-CONV']`, 57 tests, 1 failure, and `scripts/agent-preflight.sh` reports both `audit-live-rows` and `test_audit_live_rows` as FAILED. `scripts/audit-live-rows.py` alone returns rc 0 and prints `245 live rows; 1 abandoned ACTIVE`, so the count is the gate and the script's exit status does not carry it. The row is `.agents/kernel-matrix.md:143`, state `ACTIVE`, claim `CLAIM-SPEC-DFLASH2-W2`, and its implementation landed on `main` in `028438e68` (SPEC-DFLASH2 W2, [#1314](https://github.com/mudler/vllm.cpp/issues/1314), PR #1465) - a row still claimed as in-progress whose work is already reachable on `main`. NOT FIXED IN FLOW: the kernel matrix, the row and the DFlash2 claim are outside the authority of the row that found it (`LTX25-DEVICE-RESIDENCY` W0, PR #1441), and a lifecycle move owes `docs/STATUS.md`, `docs/BENCHMARKS.md` and the moved row spec's `## Now`. Found while running the full preflight for that row's gate, where it is an inherited red | bug | diff --git a/.agents/specs/ltx25-device-residency.md b/.agents/specs/ltx25-device-residency.md index b134e049f..12ccf4185 100644 --- a/.agents/specs/ltx25-device-residency.md +++ b/.agents/specs/ltx25-device-residency.md @@ -1321,11 +1321,19 @@ the two heads, which is what carrying it forward would have hidden. | M10 | re-anchored: after a chunk, re-label the decode as the writer | 1 file, +1/-1 | rc 0 | **RED**, exit 1, 1 case failed, **556 assertions, 4 failed** | exit 0, 446/446 | | N10 | detach `artifacts.frames.ppm` from the `WriteFileBytes` loop by closing it before the loop runs | 1 file, +1/-1 | rc 0 | **RED**, exit 1, 1 case failed, **554 assertions, 3 failed** | exit 0, 446/446 | -**Twelve mutations on `17eba8ce2`, the head that lands**, re-applied after the -FOURTH merge rather than carried forward. The counts moved again and the -verdicts did not: N6 reports 3 failed assertions here against 5 on `99703ce5a`, -and N1 reports 6 against 3 — the last because the per-part floor this change adds -catches the name swap as well as the boundary shift. +**Twelve mutations, run on BOTH merge heads: `17eba8ce2` and `1160d04b5`, the +head that lands.** The set was re-applied after each merge rather than carried +forward. Across the fourth merge the counts moved and the verdicts did not: N6 +reports 3 failed assertions against 5 on `99703ce5a`, and N1 reports 6 against 3 +— the last because the per-part floor this change adds catches the name swap as +well as the boundary shift. + +**Across the FIFTH merge every count is byte-identical**, all twelve of them, +which is the measurement the merge commit's argument was owed. That merge touches +`src/vt/cpu/cpu_ops.cpp`, the file whose `act(gate)` narrowing moved the counts +last time, and `git diff --numstat` reports 87 insertions and zero deletions +there. "Purely additive" was the prediction; twelve unchanged counts are the +evidence for it, and the prediction is not a substitute for the run. **SEVEN OF THE TWELVE ARE RECONSTRUCTIONS, and this is the disclosure.** Only R1b, its mirror, N1, N6 and M13 run from a definition recorded in this tree. The