Skip to content

The LTX-2.5 phase table is written by the success path only, so an aborted render and a running one both report nothing #1413

Description

@localai-bot

#1408 lands the phase table that #1010 asked for. It writes <output_dir>/phase-log.json at the end of a generation, and on the shipped default. That closes the post-mortem half of #1010 and it does not close the other half, which every full-model render this week has run into.

The table is written by the success path only. WritePhaseLog is reached after im.trace.completed = true in Ltx2VideoEngine::Generate (src/vllm/multimodal/ltx2_video.cpp:4655-4658 @ 4f3c24380). A render that is killed, aborted by a lease governor, or still running writes no table at all. Those are exactly the runs the campaign has:

  • #1375: ABORT[92] PROJECTED OVERRUN, child exit=-15, 0 frames. No table.
  • ltx25-decode-speed.md §1.3 rung 1: EXIT=137, 0 frames. Rung 2: EXIT=1, 0 frames.

Nine of the last attempts never reached a completed generation. An instrument that reports only on completion reported on none of them.

Nothing is emitted while the render runs. PhaseLog::Open and PhaseLog::Close record; they print nothing. The only stderr output is VLLM_RENDER_PHASE_LOG_STDERR=1, and it fires inside WriteJson, i.e. on the success path again. So between ltx2-gen: family=... and wrote N frames a 2.5-hour render is silent, and the two observable states — working, and hung — are byte-identical from outside.

And the unit that costs the wall has no counter. denoise is one leaf. #1375 measures ~162 s per DiT forward on the full 21.004 B model, with 60 forwards structural (30 steps x 2 CFG legs, since cfg_scale != 1.0 forces the unconditional branch — ltx2_pipeline.cpp:521-523). That is ~2.7 h inside a single leaf that reports one number, at the end, if the run survives to write it. #1375's own "What is NOT established" section names an in-process phase marker as the way to attribute that 162 s, because eu-stack unwinds zero frames inside the rc worker container and external sampling is therefore unavailable there.

What is owed

Live progress on stderr, from inside the render, on the shipped default:

  • one line when a phase opens and one when it closes, carrying elapsed and duration, so the last line printed names the phase that is currently running;
  • one line per DiT forward carrying the recipe phase, the sampler step k/N, the cumulative forward index, elapsed, and the seconds since the previous forward — which is the per-forward number LTX-2.5 full-model DiT forward measured at ~162 s on GB10; 60 forwards will not fit a 4 h lease #1375 could only get as a wall-clock interval between GPU busy/idle edges.

60 lines over 2.7 h is the right cadence: not chatty enough to drown a log, not quiet enough that a 162 s forward reads as a hang.

Not a flag. The failure happened on default settings, and a marker visible only under a flag that nobody sets would not have helped — which is the same argument #1408 makes for writing its table by default, applied to the lane that runs while the render is alive. VT_H3_PROGRESS is the existing shape for this in the tree (minimax_h3.cpp:776-793) and it is opt-in, which is why no LTX-2.5 run has one.

Precondition for attributing #1375, which is what currently caps render resolution. Stage W0 of LTX25-DEVICE-RESIDENCY (spec).

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions