[Feature] MXFP8 Dequantization VMI Lowering Issues

[jyc-coding]

Summary

Overview

Two VMI lowering issues cause 67% performance degradation (13,700 → 22,903 ticks) in
MXFP8 dequantization kernels on Ascend A5. Both are confirmed via simulator profiling
on Ascend950PR_9599 (dav_3510 model).

Test kernel: anti_mx_quant dequant, 128×256 FP8 E4M3 + E8M0 per-block scales → FP32
(32768 elements). Both CCE and DSL produce identical correct results (all elements match).

Issue 1: VMI vload lacks E2B_B32 (per-block group broadcast)

Problem

The VMI vload API supports dist_mode="brc" which lowers to BRC_B32 (single-element
broadcast: 1 element → all 64 lanes). It does not support E2B_B32 (per-block group
broadcast: 8 elements, each → 8 lanes). This forces DSL kernels to use pto.vmi.vbrc
(explicit VDUP instructions) for per-block MX scale broadcast, generating 128 extra
instructions.

Three broadcast modes on A5 hardware

Mode	Hardware behavior	Elements per load	Use case
E2B_B32	Load 8 floats, each broadcast to 8 lanes	8 → 64 (1 load)	Per-block MX scales (block=32)
BRC_B32	Load 1 float, broadcast to all 64 lanes	1 → 64 (1 load)	Per-tensor uniform scale
VDUP (vbrc)	No load, duplicate register value to all lanes	0 loads + 1 instruction	Scalar broadcast from vreg

VMI dist_mode="brc" maps to BRC_B32, not E2B_B32. The low-level
pto.vlds(dist="E2B_B32") exists in _ops.py:251 but is outside the VMI namespace
and cannot be combined with VMI vregs.

Measured impact

Approach	VDUP	Extra VLDS	Ticks	vs CCE
CCE vlds(E2B_B32)	0	0	13,700	baseline
DSL vbrc (current)	128	0	22,903	+67%
DSL dist_mode="brc" (tested)	0	+448	31,326	+129%

dist_mode="brc" is worse than vbrc because BRC_B32 broadcasts only 1 element per
load, requiring 8 separate loads per 64-lane vector (one per per-block scale). vbrc
avoids loads but emits 128 explicit VDUP instructions.

Proposed fix

Add dist_mode="e2b" (or dist_mode="group_brc") to the VMI vload API, mapping to
E2B_B32:

# Current (128 VDUP):
sc = pto.vmi.vload(scale_ptr, off, size=1)
sc_f32 = pto.vmi.vbrc(sc, size=256)

# Proposed (0 VDUP, 1 E2B_B32 load):
sc_f32 = pto.vmi.vload(scale_ptr, off, size=256, dist_mode="e2b", group=8)

This eliminates 128 VDUP instructions and aligns DSL scale broadcast with CCE.

Issue 2: VMI does not lower scf.for to RV_VLOOPv2

Problem

CCE compiles for loops to hardware RV_VLOOPv2 loops — the IFU replays the loop body
from icache with address stride baked into the loop header. Zero per-iteration scalar
setup. The VMI lowering pass does not lower scf.for to VLOOPv2 — instead it
unrolls each iteration, emitting explicit SMOVI (scalar register immediate load) and
SMOVK (scalar constant high-bit materialization) for every pto.vmi.* call.

Measured impact

Metric	CCE	DSL	Gap
RV_VLOOPv2 (hardware loops)	2	0	-2
RV_SMOVI (scalar address setup)	0	556	+556
RV_SMOVK (scalar constant materialization)	0	256	+256
Total scalar instructions	2	812	+810

CCE VLOOPv2 details:

• Loop 1 (scale decode): 4 iterations, 17 body instructions
• Loop 2 (data dequant): 64 iterations, 36 body instructions
• Hardware replays body from icache — address stride baked into VLOOP header

DSL SMOVI breakdown:

• Sd[66] (UB address register): 756 dynamic executions — offset setup per vload/vstore
• Sd[94] (secondary offset): 201 dynamic executions — scale buffer addressing
• ~9 SMOVI per data loop iteration × 64 iterations = ~576 SMOVI
• ~4 SMOVK per data loop iteration × 64 iterations = ~256 SMOVK

Each pto.vmi.vload(ptr, offset, size=N) call lowers to:

1. SMOVI Sd[66], offset — load offset into address register
2. SMOVK Sd[66], high_bits — set high bits if offset > 16-bit
3. RV_VLDS Vd, Sn[base], Sd[66] — actual vector load

In CCE, steps 1-2 are fused into the VLOOP header stride. The loop body contains only
step 3 with POST_UPDATE addressing.

• 1260 / 2832 = 44% of total dispatch slots consumed by scalar setup
• Vector instructions that could have dispatched in those slots are delayed
• The dispatch queue backs up, increasing latency between dependent vector ops

CCE's VLOOPv2 avoids this: loop body contains only vector instructions, using all 6
dispatch slots for productive work.

Cycle breakdown (DSL, from simulator profiling)

Pipe	Cycles	Description
FLOWCTRL	11,206	set_flag/wait_flag/pipe_barrier
RVECEX	8,424	Vector compute (VCVT, VMUL, VINTLV, VDUP)
RVECLD	6,190	Vector load
RVECST	5,097	Vector store
MTE3	4,643	UB → GM store
MTE2	3,189	GM → UB load
PUSHQ	2,405	VF entry/exit
SCALAR	1,589	Scalar unit (loop control, flags)
RVECSU	256	ASU execution (SMOVI/SMOVK, 1 cycle each)

Note: SCALAR (1,589) ≠ RVECSU (256). SCALAR includes the SU (main scalar unit)
doing loop control and flag management, while RVECSU is the ASU inside the vector unit.
The 812 SMOVI/SMOVK execute on the ASU (256 cycles) but their dispatch cost is hidden
in the FLOWCTRL/RVECEX/RVECLD/RVECST pipes — they delay vector dispatch, inflating
those cycle counts.

Proposed fix

Lower scf.for with constant step and compile-time-known iteration count to
RV_VLOOPv2 in the VMI lowering pass. The pass already has the loop metadata
(iteration count, step, body) — it needs to emit VLOOPv2 instead of unrolling.

This would eliminate all 812 SMOVI/SMOVK instructions, free up 44% of IDU dispatch
bandwidth for vector ops, and align DSL scalar overhead with CCE.

Motivation / use case

mxfp8 dequant

Proposed API / behavior

No response

Alternatives considered

No response

Additional context

No response