[Performance] VMI `vcvt` auto-lowering lacks `part` coordination → 1.26–2.41× slowdown vs CCE on fp32 narrowing casts

[jyc-coding]

Summary

For the pure identity narrowing cast y = cast<dst>(x) (vload → vcvt → vstore), the VMI/ptodsl lowering is correct but LSU-inefficient versus the hand-written CCE kernel. Two gaps in the auto-lowering share one root cause:

1. vcvt has no part parameter — VMI cannot express complementary part fills (P0..P3 at 4:1, EVEN/ODD at 2:1) + vor merge, so each 1/K source stream gets its own packed store instead of one contiguous store. Dominant.
2. No DINTLV_B32 lowering for vload — a full-width vlds(DINTLV_B32) (1 load, deinterleaves in-register on SLIDE) is replaced by K separate vlds(NORM) loads, adding K−1 LSU loads. Secondary.

Net effect: the DSL moves the same work from the in-register (SLIDE/ALU) units onto the bottlenecked LSU. Two reproducers show the impact scales with narrowing ratio:

Case	Narrow ratio	CCE vf	DSL vf	Gap	Bound by
Fp32ToFp8Cast256In (32×256)	4:1	306	738	+141% (2.41×)	LSU
Fp32ToFp16Cast128In (64×128)	2:1	184	231	+25.5% (1.26×)	vcvt (SLIDE)

Command line

scripts/sim_dsl.sh dsl/Fp32ToFp8Cast256In/Fp32ToFp8Cast256In_case0_fp32_fp8_32_256.py

Reproduction input

module attributes {pto.backend = "vpto", pto.target_arch = "a5"} {
  module attributes {pto.backend = "vpto", pto.kernel_kind = #pto.kernel_kind<vector>, pto.target_arch = "a5"} {
    func.func @Fp32ToFp8Cast256In_case0_fp32_fp8_32_256(%arg0: !pto.ptr<f8E4M3FN, gm>, %arg1: !pto.ptr<f32, gm>) attributes {pto.entry, pto.kernel_kind = #pto.kernel_kind<vector>} {
      %c1_i16 = arith.constant 1 : i16
      %c32_i16 = arith.constant 32 : i16
      %c0_i16 = arith.constant 0 : i16
      %false = arith.constant false
      %c0_i64 = arith.constant 0 : i64
      %c32768_i64 = arith.constant 32768 : i64
      %c1_i64 = arith.constant 1 : i64
      %c256 = arith.constant 256 : index
      %c8192_i64 = arith.constant 8192 : i64
      %c64 = arith.constant 64 : index
      %c128 = arith.constant 128 : index
      %c192 = arith.constant 192 : index
      %0 = pto.castptr %c0_i64 : i64 -> !pto.ptr<f32, ub>
      %1 = pto.castptr %c32768_i64 : i64 -> !pto.ptr<f8E4M3FN, ub>
      pto.copy_gm_to_ubuf %arg1, %0, %c0_i64, %c1_i64, %c32768_i64, %c0_i64, %c0_i64, %false, %c0_i64, %c32768_i64, %c32768_i64 : !pto.ptr<f32, gm>, !pto.ptr<f32, ub>, i64, i64, i64, i64, i64, i1, i64, i64, i64
      pto.set_flag[<PIPE_MTE2>, <PIPE_V>, <EVENT_ID0>]
      pto.wait_flag[<PIPE_MTE2>, <PIPE_V>, <EVENT_ID0>]
      pto.vecscope {
        %2 = pto.pset_b32 "PAT_ALL" : !pto.mask<b32>
        %3 = pto.pge_b32 "PAT_ALL" : !pto.mask<b32>
        scf.for %arg2 = %c0_i16 to %c32_i16 step %c1_i16  : i16 {
          %4 = arith.index_cast %arg2 : i16 to index
          %5 = arith.muli %4, %c256 : index
          %result = pto.vlds %0[%5] : !pto.ptr<f32, ub> -> !pto.vreg<64xf32>
          %6 = arith.addi %5, %c64 : index
          %result_0 = pto.vlds %0[%6] : !pto.ptr<f32, ub> -> !pto.vreg<64xf32>
          %7 = arith.addi %5, %c128 : index
          %result_1 = pto.vlds %0[%7] : !pto.ptr<f32, ub> -> !pto.vreg<64xf32>
          %8 = arith.addi %5, %c192 : index
          %result_2 = pto.vlds %0[%8] : !pto.ptr<f32, ub> -> !pto.vreg<64xf32>
          %9 = pto.vcvt %result, %2 {part = "P0", rnd = "R", sat = "SAT"} : !pto.vreg<64xf32>, !pto.mask<b32> -> !pto.vreg<256xf8E4M3FN>
          %10 = pto.vcvt %result_0, %2 {part = "P0", rnd = "R", sat = "SAT"} : !pto.vreg<64xf32>, !pto.mask<b32> -> !pto.vreg<256xf8E4M3FN>
          %11 = pto.vcvt %result_1, %2 {part = "P0", rnd = "R", sat = "SAT"} : !pto.vreg<64xf32>, !pto.mask<b32> -> !pto.vreg<256xf8E4M3FN>
          %12 = pto.vcvt %result_2, %2 {part = "P0", rnd = "R", sat = "SAT"} : !pto.vreg<64xf32>, !pto.mask<b32> -> !pto.vreg<256xf8E4M3FN>
          pto.vsts %9, %1[%5], %3 {dist = "PK4_B32"} : !pto.vreg<256xf8E4M3FN>, !pto.ptr<f8E4M3FN, ub>, !pto.mask<b32>
          pto.vsts %10, %1[%6], %3 {dist = "PK4_B32"} : !pto.vreg<256xf8E4M3FN>, !pto.ptr<f8E4M3FN, ub>, !pto.mask<b32>
          pto.vsts %11, %1[%7], %3 {dist = "PK4_B32"} : !pto.vreg<256xf8E4M3FN>, !pto.ptr<f8E4M3FN, ub>, !pto.mask<b32>
          pto.vsts %12, %1[%8], %3 {dist = "PK4_B32"} : !pto.vreg<256xf8E4M3FN>, !pto.ptr<f8E4M3FN, ub>, !pto.mask<b32>
        }
      }
      pto.set_flag[<PIPE_V>, <PIPE_MTE3>, <EVENT_ID0>]
      pto.wait_flag[<PIPE_V>, <PIPE_MTE3>, <EVENT_ID0>]
      pto.copy_ubuf_to_gm %1, %arg0, %c0_i64, %c1_i64, %c8192_i64, %c0_i64, %c8192_i64, %c8192_i64 : !pto.ptr<f8E4M3FN, ub>, !pto.ptr<f8E4M3FN, gm>, i64, i64, i64, i64, i64, i64
      return
    }
  }
}

Expected performance

The CCE casts K deinterleaved streams into complementary quarters/halves of one destination register, then vor-merges → one contiguous NORM store:

• 4:1 (fp8): vcvt(PART_P0), vcvt(PART_P1), vcvt(PART_P2), vcvt(PART_P3) → vor×3 → 1× vsts(NORM_B8).
• 2:1 (fp16): vcvt(PART_EVEN), vcvt(PART_ODD) → vor×1 → 1× vsts(NORM_B32).

Actual performance

VMI's pto.vmi.vcvt(src, to_dtype=…) exposes no part argument, so the lowering fills the same part each time — the resulting sparse registers are overlapping, not complementary, and cannot be vor-merged. Each stream therefore gets its own packed store:

• 4:1: 4× pto.vcvt {part="P0"} → 4× pto.vsts {dist="PK4_B32"} (4 stores vs 1, +3/row).
• 2:1: 2× pto.vcvt {part="EVEN"} → 2× pto.vsts {dist="PK_B32"} (2 stores vs 1, +1/row).

Profiling data (optional)

No response

Git commit

PTOAS 0.59