feat(LTX25-A2VID-RECIPE): the audio-to-video recipe, and the phase field a boolean could not express (#1117)

[localai-bot]

A2VidPipelineTwoStage (a2vid_two_stage.py:53 @ fd4ded7f) had no recipe row,
so pipeline_kind = a2vid_two_stage got the generic table refusal naming the
pair rather than the missing machinery. #922
is CLOSED and closed the audio conditioning, not the recipe: a supplied take
rode distilled_two_stage, which
ltx25-a2v-audio-input.md:438-446
already records as a different trajectory. A closed issue is not evidence.

Issue #1117. Spec
.agents/specs/ltx25-a2vid-recipe.md,
committed at d38ab8c86, before any code.

What the recipe is, field by field

Four fields differ from the recipe a take rides today, and each of them
renders — a finished clip at the right size, frame count and sample rate.

Field	a2vid	distilled_two_stage	Upstream
stage 1 sigmas	DERIVED from the step count	the frozen 9-value distilled list	self._scheduler.execute(steps=num_inference_steps), :225-227
stage 1 guidance	the params table's video row, caller-overridable	fixed, override refused	MultiModalGuider(params=video_guider_params, ...) :230-240, fed from utils/args.py:947-1006
stage 1 stepper	plain Euler	kEulerAncestral on 2.5	:229-258 passes no stepper, so EulerDiffusionStep() applies, utils/blocks.py:526-527
stage 1 AUDIO guider	the DEFAULT positive-only params	n/a	MultiModalGuiderParams() at :237-239, ltx-core components/guiders.py:200-210

The audio guider is the field a reader is most likely to "fix" by symmetry with
OneStagePhase, which takes the table's row and is right to
(ti2vid_one_stage.py:215-218). A2Vid does not: the stream it would guide is
frozen, so the table's cfg_scale = 7.0 would buy an unconditional forward and a
negative text encode for a delta multiplied into a latent the sampler cannot
move.

Stage 2 is upstream's STAGE_2_DISTILLED_SIGMAS (:164), the spatial upsample
(:261), noise_scale = stage_2_sigmas[0] (:288) and SimpleDenoiser (:278).

Four rows — 2, 2.3, 2.4, 2.5 — mirroring the t2a_one_stage rows one for one
and for the same reason: the pipeline takes whatever resolve_cli_params() read
off the checkpoint (:311, against t2a_one_stage.py:178-179).

Two things upstream makes REQUIRED, refused by name

• --audio-path is required=True (:312-317). Checked at generate time,
  because pipeline_kind is a load knob and audio_path is a per-generation
  extra, so the question is only decidable once a request exists. Without the
  take the soundtrack is GENERATED and the clip looks finished.
• --distilled-lora is required=True (utils/args.py:1140-1153). Checked
  at load. Without the adapter, stage 2's three-sigma refinement runs on weights
  that were never distilled for it.

Both are keyed on a recipe flag rather than a pipeline_kind string compare, for
the reason audio_only already gives in the header. The second flag has a second
user waiting: ti2vid_two_stages (#1093)
and keyframe_interpolation (#1096)
select the same parser.

The phase field a boolean could not express

Ltx2PhaseDenoiser { kGuided, kSimple } is new, and it exists because
allow_guidance_override cannot describe a2vid's stage 2.

That boolean answers "does this pipeline's CLI carry the guider flags at all".
distilled.py selects default_2_stage_distilled_arg_parser
(utils/args.py:1188), which never adds them, so an override there names a knob
the pipeline has no surface for and the engine refuses it — correctly, and that
refusal is landed and gated. a2vid_two_stage.py:311 selects
default_2_stage_arg_parser, which DOES carry them, and they reach stage 1's
guider alone (:233-236) because stage 2 constructs SimpleDenoiser and takes
no params.

Neither value of the boolean says that. false rejects a request upstream
accepts. true applies the override to stage 2's positive-only params and
switches on a guidance pass upstream does not run — invisibly, since an extra
forward changes no output shape, frame count or sample rate. So a2vid's stage 2
is allow_guidance_override = true and kSimple, and the skip is tested
AFTER the refusal: every recipe that refuses today is kSimple too, so the other
order would turn three landed refusals into silent ignores.

Reachability

Entry point: LoadVideoEngine with pipeline_kind = a2vid_two_stage — a
documented value of a documented load extra — then Generate with the
audio_path extra. The chain is include/vllm.h -> src/capi/vllm_c.cpp ->
Ltx2VideoEngine::Generate -> the a2vid dispatch row -> the phase loop ->
Ltx2GuidedDenoise. ltx2-gen --pipeline-kind a2vid_two_stage --audio-path is
the same two calls through the ABI. No test constructs a recipe, a guider, a
phase or a modality by hand.

M1 is the reachability mutation: deleting the dispatch row REDs the whole
case at the load.

What does NOT reach it, stated rather than left to be found. /v1/videos
cannot drive this. VideoGenParamsFromRequest (video_engine.cpp:349-384) never
writes gen.extras, so no per-generation extra reaches any engine over HTTP
(#928). pipeline_kind IS a
load extra and reaches a server through --video-extra, and every request to
such a server is then refused for the missing take. Reachable from
include/vllm.h and from ltx2-gen; not over HTTP.

RED before green

Both binaries, with the dispatch row absent and everything else in place:

test_ltx2_pipeline  44 cases | 42 passed | 2 failed;  2498 assertions | 1 failed;  exit 1
test_ltx2_video     74 cases | 72 passed | 2 failed;  2191 assertions | 4 failed;  exit 1

ERROR: test case THREW exception: Unsupported LTX pipeline kind/version:
'a2vid_two_stage'/'2.5'. Recipes are resolved from an EXACT (kind, version)
table ... and never defaulted: a plausible-but-wrong sigma schedule or guidance
scale renders a video rather than failing.

Green on the same tree with the row present: test_ltx2_pipeline 44/44, 2598
assertions, exit 0; test_ltx2_video 74/74, 2275 assertions, exit 0.

The take is CONSUMED, not carried

A recipe assertion proves frozen = True and noise_scale = 0.0 are SET. These
say the DiT saw the consequence, read off the LAST phase — so stage 2's own
noise_scale of 0.909375, which the loop applies to both streams, is inside what
they measure:

• audio_frozen, derived from the denoise mask the loop uses and read AFTER the
  noiser;
• audio_sigma_max == 0.0, the scalar Modality.sigma half of upstream's
  frozen (utils/types.py:104-106), which the mask cannot reach;
• the latent's digest is bit-identical across seeds — it is the encoded file
  and not a sample — and moves with the window, so the first control cannot
  be passing on a constant.

The guidance arms, in x0 space, on all four

Stage 1's guider is cfg 3.0 / stg 1.0 / rescale 0.7 / modality 3.0, so all four
passes run and the rescale branch — the one term that is not invariant between
the two spaces — is live. x0 == latent - sigma*velocity is asserted per arm.
M11's RED, verbatim:

arm = cond        max|x0 - (latent - sigma*v)| = 1.64497   max|x0 - velocity| = 0
arm = uncond      max|x0 - (latent - sigma*v)| = 1.64082   max|x0 - velocity| = 0
arm = perturbed   max|x0 - (latent - sigma*v)| = 1.64939   max|x0 - velocity| = 0
2 cases | 1 passed | 1 failed;  94 assertions | 8 failed;  exit 1

The stage-2 skip has no trace field to read, so it is measured on artifact bytes:
two renders whose only difference is video_stg_scale = 1.0, which is ALREADY
stage 1's own value (utils/constants.py:52) and is 0.0 on stage 2. Equal bytes
mean the override stopped at stage 1. Both recipe values are REQUIREd first, so
a table change turns the comparison into a failure rather than a tautology.

Mutations

Focused gate ./build/tests/test_ltx2_video --test-case='ltx2 a2vid:*' — the
filter is admissible only because the harness asserts it matched exactly 2 cases
and a non-zero assertion count; neither case name contains a comma. Recipe-shape
mutations also run the whole test_ltx2_pipeline binary.

#	Mutation	git diff --stat	BUILT	: error:	exit	counts	verdict
M1	the a2vid dispatch row deleted (reachability)	ltx2_pipeline.cpp | 2 +-	YES	0	1	2c/9a	DETECTED
M2	stage 1 spatial_downscale 2 -> 1	ltx2_pipeline.cpp | 2 +-	YES	0	1	2c/19a	DETECTED
M3	stage 1 given the frozen distilled sigmas	ltx2_pipeline.cpp | 2 +-	YES	0	1	2c/94a	DETECTED
M4	stage 1 allow_guidance_override -> false	ltx2_pipeline.cpp | 2 +-	YES	0	1	2c/19a	DETECTED
M5	stage 1 audio guider taken from the params table	ltx2_pipeline.cpp | 2 +-	YES	0	1	2c/94a	DETECTED
M6	stage 1 stepper -> kEulerAncestral	ltx2_pipeline.cpp | 2 +-	YES	0	0 video / 1 pipeline	2c/94a; 44c/2598a	DETECTED by the RECIPE case only
M7	the required-take check never fires	ltx2_video.cpp | 2 +-	YES	0	1	2c/92a	DETECTED
M8	the required-adapter check never fires	ltx2_video.cpp | 2 +-	YES	0	1	2c/91a	DETECTED
M9	the frozen stream's denoise mask left at 1	ltx2_video.cpp | 2 +-	YES	0	1	2c/94a	DETECTED
M10	the frozen scalar sigma left at the schedule's	ltx2_video.cpp | 2 +-	YES	0	1	2c/94a	DETECTED
M11	every guidance arm left in VELOCITY space	ltx2_video.cpp | 3 +--	YES	0	1	2c/94a, 8 failed	DETECTED
M12	the kSimple skip deleted, so the override reaches stage 2	ltx2_video.cpp | 2 +-	YES	0	1	2c/94a	DETECTED

M6 is stated as it measured, not as it would read better. The stepper is a
recipe field and only the recipe case sees it; the end-to-end case has no
baseline to compare a trajectory against, and this recipe's
noise_seed_offset is 0 so the ancestral arm changes no digest the trace carries.

One harness defect, found and fixed in flow. The stage-2 artifact comparison
was CHECK(a == b) over PPM pixels and a WAV. A failing one dumps raw bytes into
the report, and that killed the mutation harness with a UnicodeDecodeError
between applying M12 and restoring it — the exact shape #922's spec records. The
finally restored the tree, the comparison is now a differing-byte COUNT
(99c4cf811), and the harness decodes with errors="replace".

Gate

cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=Release -DVLLM_CPP_CUDA=OFF
cmake --build build -j6 && ctest --test-dir build -j4 --output-on-failure

On the merged tree at 85e562d1d, origin/main = 589abad12, CLEAN rebuild
(rm -rf build first, because 589abad12 adds headers and an incremental build
masks -Werror):

CONFIGURE_EXIT	0
BUILD_EXIT	0
: error: in the build log	0
ctest -N	Total Tests: 502
CTEST_EXIT	0
pass/fail	100% tests passed, 0 tests failed out of 502 (2 skipped: test_modelopt_mixed_precision_checkpoint, test_voxtral_e2e)
No space left / BFD in the build log	0 and 0 over 1475 lines, against a positive control file where the same two greps return 1 and 1
load average	99.37 at configure, 41.20 entering ctest, 83.82 after
free disk	30G of 447G before, 20G after; the build tree is removed on completion

502, not 501: 589abad12 adds test_ops_conv1d_general. Nothing known-red
fired — test_async_llm (#294), test_engine_core_proc (#1052),
test_serve_low_tools (#428) and test_cpu_x86_llamacpp_floor (#618) are all
load-dependent and all passed; windows-msvc-* (#584) has no main baseline and
does not run here.

Both ltx2 binaries were also run WHOLE, with no --test-case filter, and their
case and assertion counts asserted non-zero rather than read as "no failures":
test_ltx2_video 74 cases / 2276 assertions / Status: SUCCESS! / exit 0, and
test_ltx2_pipeline 44 / 2598 / SUCCESS! / exit 0. Status: is grepped beside
the counts because a thrown doctest case prints 0 failed next to FAILURE!,
and the exit code is taken directly rather than after a pipe.

What is NOT claimed

No render on real weights, and no oracle-run comparison. The GPU is out of
bounds for this row and dgx.casa is network-alive with an sshd that will not
handshake. vllm-omni is UNPINNED (#633)
and carries no LTX-2.5 recipe; no LTX-2.5 checkpoint here has a recorded sha256
(#1048). Every value is read off
upstream SOURCE at fd4ded7f and every number is measured on this tree's own
reduced fixture.

The adapter placement diverges and cannot be repaired from the request.
stage_2_loras = (*loras, *distilled_lora) (:114) puts the distilled adapter on
stage 2 ALONE, against loras=tuple(loras) for stage 1 (:107). This engine
fuses at load into ONE weight set — ltx2_video.cpp:816-820 is the only
dit_options.loras.push_back in the tree — so stage 1 sees it too. That changes
the trajectory, so it changes the PIXELS: the frames this renders are not the
ones upstream renders for the same checkpoint, take and seed. What it leaves
untouched is the frame count, the shapes, the sample rate and the errors —
nothing in the SHAPE of the result says anything is wrong. It is therefore
#1118 and ## Owed rather than
a comment. It bounds #1093 and #921, which need the same seam.

It is not undetectable, and an earlier draft of this paragraph said it was.
"Changes nothing a caller can read" is false — a caller reads pixels — and it is
false in the direction that hides work, because it implies no instrument could
ever see this. The instrument that WOULD see it is the real-weights comparison
against upstream's own render that the section above records this row as not
having: same checkpoint, same take, same seed, upstream's stage 1 on the base
weights against ours on base + distilled. That comparison is owed, not
impossible, and #1118 owns it. docs/USAGE.md tells a reader of the a2vid
command the same thing in the same change.

Two anchors in the dispatch that started this row were wrong and are corrected
rather than propagated. The a2v guidance default is 3.0, at
utils/args.py:986-995 through utils/constants.py:54, not 0.0 at
a2vid_two_stage.py:318-323 — which is --audio-start-time. And the
AudioConditioner is built at :96-102 and called at :200, not :53/:143.

Six anchors of this row's own, corrected after review. No gate protects an
anchor here (#632), so each was
re-derived from the sentence making the claim rather than by reading the cited
span:

Was	Is	How it was checked
--a2v-guidance-scale at utils/args.py:987-996	:986-995	parser.add_argument( opens at :986, the name is at :987, the default at :989, the closer at :995. The old span started at the name and ran one line past
default_2_stage_distilled_arg_parser at utils/args.py:1187	:1188	:1187 is blank
the six video-guider flags at utils/args.py:947-1006	:947-1006	--video-cfg-guidance-scale at :948 through --video-skip-step at :997-1006; the audio group starts at :1007. The old span stopped at the last flag's opening line
"stage 1's 40-step schedule"	30 on this row's 2.5	LTX_2_3_PARAMS sets num_inference_steps=30 (utils/constants.py:85), LTX_2_4_PARAMS inherits it (:124), and 2.5 resolves onto that row. 40 is 2.0's own default (:47)
"the params row names block 29"	28	LTX_2_3_PARAMS overrides 2.0's [29] to [28] (:86), inherited the same way
"all four the params table distinguishes"	four keys of OUR table	_PARAMS_SINCE_VERSION (:130-133) has two rows, (2,4) and (2,3), falling through to LTX_2_PARAMS at :179. Ltx2DetectPipelineParams mirrors that and already said so

Neither number was load-bearing — the code reads params.num_inference_steps and
the tests assert relatively — which is exactly why nothing caught them. The §2
table also listed LightricksNegativePrompt() flat where the dispatch uses
kOmniNegativePrompt on 2 and 2.3; that split is which reference owns the
row, and t2a_one_stage splits identically at the same four versions.

include/vllm/multimodal/ltx2_video.h:473 carries the same 987-996 span, but
it is on main and predates this row, so it is reported rather than rewritten
here.

This row's own insertions staled anchors elsewhere, and that is reported rather
than swept. Two are repaired here because this row owns them, and both now name
the construct instead of a line range: the spec pointed at
ltx2_pipeline.cpp:1464-1471 for the t2a_one_stage rows, which the F4 comment
above them moved in this very commit, and docs/USAGE.md pointed at
ltx2_pipeline.cpp:1306-1313 for the DFR refusal, which this row's 145 inserted
lines moved to :1434-1441.

FIVE more docs/USAGE.md coordinates moved the same way, in sections this row
does not own. Each was re-derived by matching main's content at the cited line
against this tree:

Cited at	Was	Is now
USAGE.md:918	ltx2_video.cpp:2900	:2968
USAGE.md:1170	ltx2_video.cpp:1007-1012	:1040-1045
USAGE.md:1174	ltx2_video.cpp:1955-1990	:1998-2033
USAGE.md:1176	ltx2_video.cpp:1991-2004	:2034-2047
USAGE.md:2672	ltx2_pipeline.h:768-803	:848-883

They are NOT rewritten here. That page states its own convention — "read an
unpinned coordinate as unverified" — and its three ltx2_video.cpp coordinates
that sit beside a @ b5756ea8c pin are implicitly pinned by it and are correctly
unchanged. Its last re-derivation was a scoped unit of work with its own review,
not a rider on someone else's row, and this one should be too.

Two things checked and found NOT to be this row's doing: USAGE.md:2644 cites
ltx2_video.cpp:377-383 for kKnownLoadExtras, which is at :378-385 on main
and on this branch alike, so that one is pre-existing (the same passage already
names #1097 for stale text
beside that array).

Merge hazard. The READER ANCHORS list in ltx2_video.cpp is re-derived in
this change (798 808 809 871 967 983 1018 1109 1134 1239 1280 1322 1324). A
clean git merge that inserts a line above it will not warn, and
test_ltx2_video will.

It was re-derived once more after merging 589abad12, with the same walk the
test uses, and is unchanged — that commit adds vt::Conv1d and touches no
LTX-2.5 file. The deriving instrument was armed before it was believed: one
inserted line above the anchors moves all thirteen and it reports MISMATCH,
exit 1, where the real tree reports MATCH, exit 0. A checker that reads its own
expectation out of the file it checks is a tautology unless you make it fail
(#911).

.agents/issue-index.md was taken deliberately rather than by the union driver:
it is untouched by 589abad12, and it is verified against that revision anyway —
byte-identical prefix, no base row changed or dropped, every id unique, this
branch appending exactly #1117 and #1118. That check was armed too: editing
a base row trips the prefix and row halves, and appending a duplicate id trips
the third.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [claude-code]