feat(MODEL-MUSIC-MUSIC3): the 2.4B fp32 DiT onto the device, staged once (#672)

[localai-bot]

feat(MODEL-MUSIC-MUSIC3): the 2.4B fp32 DiT onto the device, staged once (#672)

FOLLOWING_AGENTS_PROTOCOL

Spec §11.4 recorded three device rows as owed. §12 closed the
arm-independent one. This closes the DiT, which is the one that
mattered: at a real duration it is not one stage among six, it is the
request. A 45 s clip at the shipped defaults runs DitForward 660
times (30 steps x 2 CFG branches x 11 windows) for roughly 634 TFLOP
against ~29 TFLOP for the entire autoregressive half — about 20x
everything else in the model put together. That is why d9441ef3's
device arm reached only 0.946x: it moved the 8.6B language model and
left the stage twenty times larger on the host.

What moved, and onto what

No new kernel. Every op already existed with a CUDA provider; this
adds a forward that composes them.

reference helper	shared op
Linear	vt::MatmulBT (+ vt::Add for the rank-1 bias)
LayerNorm	vt::LayerNorm
ApplyPartialRotary	vt::RopeFromCache
Attention (NON-causal)	vt::AttentionCross, bias nullptr
value * silu(gate)	vt::SiluAndMul, over a stage-time half swap
PointwiseConv (both 1x1)	vt::MatmulBT on the transposed activation

The 1x1 convolutions are GEMMs, and that is what unblocked the row.
vt has no CUDA 1-D convolution provider — the finding §11.4 recorded
against the vocoder applies to the DiT's preprocess_conv and
postprocess_conv too. But conv(x)^T[t][co] = SUM_ci x^T[t][ci] * W[co][ci] = MatmulBT(x^T, W), so the forward works FRAME-MAJOR
throughout and transposes once on the host at each end, where the
tensors are [128, length]. Nothing is hand-rolled outside the seam.

fp32 stays fp32. Spec §2.1: the acoustic half is float32 by
upstream's choice. Every staged weight and device activation is kF32.

Correctness — same goldens, same bounds, nothing widened

The CPU arm is bit-identical structurally, not by measurement.
minimax_music3_acoustic.cpp, minimax_music3_ar.cpp,
minimax_music3_llm.cpp and vocoder1d.cpp have a ZERO DIFF.
--speech-device 0 takes the same DitForward, source byte for source
byte, so there is no number to move. The device forward is an additional
entry point in a new file, the shape minimax_h3_device.cpp already uses.

Reduced dimensions, vs upstream's own goldens, at the EXISTING bound
(kRelTol 1e-5 / kAbsFloor 1e-6), each case reporting BOTH arms'
distance to the golden:

arm	worst |arm - upstream|
host DitForward (the accepted control)	1.565e-07
device forward, CPU backend	1.192e-07
device forward, CUDA sm_110	2.980e-07

FULL SCALE, the real 2.4B checkpoint vs the committed oracle capture on
thor:gpu0, 11 008 values/step, bounds unchanged (1e-4 / 5e-5 / 5e-6):

arm	step	bit-identical	mean|d|	max|d|	outside
Thor CPU	first	423 (3.843%)	1.71434e-06	2.38419e-05	0
Thor CPU	last	235 (2.135%)	2.22396e-06	2.83718e-05	0
Thor CUDA	first	473 (4.297%)	1.64344e-06	2.47955e-05	0
Thor CUDA	last	222 (2.017%)	2.44677e-06	2.59876e-05	0
CONTROL torch-vs-torch	first	15.416%	7.526e-07	7.153e-06	—
CONTROL torch-vs-torch	last	5.596%	1.424e-06	1.335e-05	—

The device arm sits ON TOP of the host arm — better on two of four
figures, marginally worse on the other two — and both sit at the same
multiple of the recorded torch-vs-torch control. The Thor CPU arm
reproduces the x86-64 numbers this spec already recorded VALUE FOR
VALUE, so the CPU path is unchanged across two architectures.

Two mutations, because a bound nothing violates has not been shown to
discriminate. Pre-swapping the ff_in halves makes the stage-time swap
undo the test's, so the forward computes silu(value) * gate: 20 of 20
values outside the bound, worst |diff| 1.538e-03, four orders above the
noise. And the conditional/unconditional branches must differ: 20 of 20
do, on both backends.

Speed — thor:gpu0 (NVIDIA Thor, sm_110), per DiT forward

Named because a number without its device is meaningless across this
fleet; nothing here is compared to a dgx:gpu0 or orin:gpu0 number.
VLLM_CPP_MUSIC3_DIT_REPEAT=R times ONLY the forward loop — the 9.7 GB
load, the golden reads and the staging are outside it.

arm	repeats	fwd	loop	per forward	staging	load
CPU	1	4	819.818584 s	204.954646 s	no-op	3.42
CPU	1	4	819.992 s	204.998 s	no-op	10.37
CUDA	1	4	0.749077 s	0.187269 s	0.603561 s	4.79
CUDA	3	12	2.110301 s	0.175858 s	0.660600 s	5.32
CUDA	1	4	0.743367 s	0.185842 s	0.609463 s	5.1
CUDA	1	4	0.743881 s	0.185970 s	0.612787 s	4.44

Fit: slope 0.170607 s/forward, intercept 0.063012 s. The device R=1
point was taken THREE times across two sessions, bracketing R=3, at
0.749077 / 0.743367 / 0.743881 s — 0.77 % spread. 204.955 s host vs
0.1706-0.1873 s device: 1102x on the matched pair, 1201x on the slope.

The contention asymmetry was measured away, not argued away. The
first CPU point sat at load 10.37 against the device arm's 4.4-5.3, which
would have inflated the ratio if it mattered. Re-taken on an idle box
(load 3.42) with the fixed instrument it reads 204.954646 s against
204.998 s, 0.021 %: the host DiT forward is single-threaded on 14 cores,
so a load of 10 still leaves it a core. Both points are reported.

The weights are staged ONCE, as a measurement. One staging costs
0.60-0.66 s; the entire FOUR-forward loop costs 0.745 s and the TWELVE-
forward loop 2.110 s, where twelve stagings would be 7.35 s alone. The
loop's intercept is a tenth of one staging. A per-forward upload is
arithmetically excluded.

Whole-process, which is lower and is the honest ceiling on what a user
sees today: 1054-1071 s vs 238-298 s (3.5-4.5x) for the same binary
including the identical NAS load, the spread being NAS cache state rather
than compute; the full two-arm correctness series, 49 min 17 s vs
15 min 49 s (3.12x). The distance between 1100x on the DiT and 4x on
the process IS the owed list.

No e2e song pair, and that is a limit not an omission: at 30 steps
the host DiT alone extrapolates to ~37.6 h, and at a setting short enough
to run, the pair would be measuring the vocoder. No parity claim —
SGLang-Omni is gateable = no and every reference axis stays PENDING.

One instrument defect, found inside this change

The first timing line printed DIT_TIMING arm=1 on the CPU run: a
const char* in a doctest MESSAGE chain takes the bool conversion.
That is #672's OWN §11.5 defect reappearing in a new line — the lesson
was written down and a fresh << chain reintroduced it. Both lines are
now assembled as one std::string. EVERY number was then re-taken with
the fixed instrument, and the CPU arm's pre-fix point is kept BESIDE its
post-fix twin rather than replaced by it, because the pair is what proves
the label defect never touched the values: 819.992 s against 819.818584 s.

What is still OWED, and a correction to §11.4

§11.4 said the depth decoder was blocked on "nothing but the work".
That is wrong, and this row found it out. The depth decoder and the
condition mix run at ArCompute::kBFloat16, which rounds the RESULT of
every op to bf16 (minimax_music3_ar.cpp:36-40). Routing them through an
f32 vt::MatmulBT would silently drop that rounding — a change to the
numbers wearing a refactor's clothes. Mirroring them needs bf16 STORAGE,
a dtype decision with its own evidence. They are also ~15 TFLOP against
the DiT's 634. The vocoder row is unchanged: vt still has no
ConvTranspose1d, and that op has three consumers.

minimax_music3_device is added to the merged-GEMM allowlist with a
reason rather than folded: ff.net.0.proj is ALREADY one merged
nn.Linear and ALREADY one MatmulBT, so a merged-GEMM seam has nothing
left to merge, and layers::UnquantizedMlpGateUpMethod is bf16-only,
bias-free and OwnedTensor-resident — three shared-layer changes, not a
model fold.

Lease discipline, recorded because it was imperfect

The CUDA build, the correctness series and the first timing runs were
driven over ssh under flock $HOME/gpu.lock — this row's brief, since
superseded by rc. During that window the fleet reported thor:gpu0
FREE while it was in use. The first device series ran under a real
rc hold (a91d21dc, 10:32Z-10:54Z) that carried no --reason and a
75 m TTL for ~22 m of work; both are errors. The matched pair above ran
under 8aa5cd6d with a reason string and a 40 m TTL, released early at
23 m on completion. rc run cannot serve this particular job because the
build lives on the device host's filesystem and rc run's container sees
only /workspace; a shell is the sanctioned case for a hold.

Thor's uptime is unbroken across every arm (system boot Jun 5 15:36;
up 2 days, 14:37 at the first arm through 16:56 at the last), so no
pair straddles a restart, and no number here is compared to one from
dgx:gpu0 (GB10) or orin:gpu0 (Orin).

Gates

Local x86-64, non-zero assertion counts: test_minimax_music3_acoustic
32/283 (was 27/265), test_minimax_music3_speech 9/223,
test_minimax_music3_ar 26/352, test_minimax_music3_loader 21/1393,
test_minimax_music3_quant 29/125, test_speech_engine 11/38,
test_capi 65/653, test_speech_api 6/67, test_openai_api_server
62/733. With CHECKPOINT_ROOT set: test_minimax_music3_acoustic_real
6/76, test_minimax_music3_ar_real 4/894,
test_minimax_music3_quant_real 6/319, test_minimax_music3_llm_real
4/220 — every one matching its recorded count.

Thor CUDA build: test_minimax_music3_acoustic 32/291 (8 more than
x86 — the CUDA device case RUNS instead of skipping),
test_minimax_music3_acoustic_real 6/985 (device 0) and 6/988
(device 1), test_minimax_music3_speech 9/223, test_speech_engine
11/37 (one fewer BY DESIGN on a CUDA build).

Issue: #672
Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]