[Performance] VMI `vcvt` has no `part` → ExpertTokenHist 4:1 si32→ui8 cast over-computes quarters, +27.7% vs CCE

[jyc-coding]

Summary

ExpertTokenHistVfKernel (MoE expert-token histogram) is a CCE-isomorphic DSL port whose only meaningful divergence is the si32→ui8 4:1 narrowing cast feeding dhistv2. The CCE emits one vcvt(si32→ui8, PART_P0); the VMI/ptodsl lowering cannot target a single part, so it deinterleaves the input into sub-streams, casts multiple parts (P0..P3), and vor-merges them back into a full 256-element ui8 register — but dhistv2 is masked to VL64 (quarter 0 only), so 3 of the 4 part-casts and the entire merge tree produce data that is never histogrammed.

Case (case2, Count=512)	CCE	DSL	Gap
vf_real_execute_time	173	221	+27.7% (1.28×)
V-pipe instr_num (PUSHQ VF)	39	52	+13
VLOOP body	5 instr/iter	13 instr/iter	+8/iter

Command line

scripts/sim_dsl.sh dsl/ExpertTokenHistVfKernel/ExpertTokenHistVfKernel_case0_u8_u32_256.py

Reproduction input

module attributes {pto.backend = "vpto", pto.target_arch = "a5"} {
  module attributes {pto.backend = "vpto", pto.kernel_kind = #pto.kernel_kind<vector>, pto.target_arch = "a5"} {
    func.func @ExpertTokenHistVfKernel_case0_u8_u32_256(%arg0: !pto.ptr<ui32, gm>, %arg1: !pto.ptr<si32, gm>) attributes {pto.entry, pto.kernel_kind = #pto.kernel_kind<vector>} {
      %c1_i16 = arith.constant 1 : i16
      %c3_i16 = arith.constant 3 : i16
      %false = arith.constant false
      %c0_i64 = arith.constant 0 : i64
      %c1024_i64 = arith.constant 1024 : i64
      %c1_i64 = arith.constant 1 : i64
      %c64 = arith.constant 64 : index
      %c0_i16 = arith.constant 0 : i16
      %c0 = arith.constant 0 : index
      %c192 = arith.constant 192 : index
      %c0_i32 = arith.constant 0 : i32
      %c1_i32 = arith.constant 1 : i32
      %c128 = arith.constant 128 : index
      %0 = pto.castptr %c0_i64 : i64 -> !pto.ptr<si32, ub>
      %1 = pto.castptr %c1024_i64 : i64 -> !pto.ptr<ui32, ub>
      pto.copy_gm_to_ubuf %arg1, %0, %c0_i64, %c1_i64, %c1024_i64, %c0_i64, %c0_i64, %false, %c0_i64, %c1024_i64, %c1024_i64 : !pto.ptr<si32, gm>, !pto.ptr<si32, ub>, i64, i64, i64, i64, i64, i1, i64, i64, i64
      pto.copy_gm_to_ubuf %arg0, %1, %c0_i64, %c1_i64, %c1024_i64, %c0_i64, %c0_i64, %false, %c0_i64, %c1024_i64, %c1024_i64 : !pto.ptr<ui32, gm>, !pto.ptr<ui32, ub>, i64, i64, i64, i64, i64, i1, i64, i64, i64
      pto.set_flag[<PIPE_MTE2>, <PIPE_V>, <EVENT_ID0>]
      pto.wait_flag[<PIPE_MTE2>, <PIPE_V>, <EVENT_ID0>]
      pto.vecscope {
        %2 = pto.pset_b32 "PAT_ALL" : !pto.mask<b32>
        %3 = builtin.unrealized_conversion_cast %c0_i16 : i16 to ui16
        %4 = pto.pset_b16 "PAT_ALL" : !pto.mask<b16>
        %5 = pto.vdup %3, %4 : ui16, !pto.mask<b16> -> !pto.vreg<128xui16>
        %6 = pto.pset_b8 "PAT_ALL" : !pto.mask<b8>
        %7 = pto.pset_b8 "PAT_VL64" : !pto.mask<b8>
        %8 = pto.pand %7, %7, %6 : !pto.mask<b8>, !pto.mask<b8>, !pto.mask<b8> -> !pto.mask<b8>
        %9:2 = scf.for %arg2 = %c0_i16 to %c3_i16 step %c1_i16 iter_args(%arg3 = %5, %arg4 = %5) -> (!pto.vreg<128xui16>, !pto.vreg<128xui16>)  : i16 {
          %27 = arith.index_cast %arg2 : i16 to index
          %28 = arith.muli %27, %c64 : index
          %result_12 = pto.vlds %0[%28] : !pto.ptr<si32, ub> -> !pto.vreg<64xsi32>
          %low_13, %high_14 = pto.vdintlv %result_12, %result_12 : !pto.vreg<64xsi32>, !pto.vreg<64xsi32> -> !pto.vreg<64xsi32>, !pto.vreg<64xsi32>
          %low_15, %high_16 = pto.vdintlv %low_13, %low_13 : !pto.vreg<64xsi32>, !pto.vreg<64xsi32> -> !pto.vreg<64xsi32>, !pto.vreg<64xsi32>
          %low_17, %high_18 = pto.vdintlv %high_14, %high_14 : !pto.vreg<64xsi32>, !pto.vreg<64xsi32> -> !pto.vreg<64xsi32>, !pto.vreg<64xsi32>
          %29 = pto.vcvt %low_15, %2 {part = "P0", sat = "NOSAT"} : !pto.vreg<64xsi32>, !pto.mask<b32> -> !pto.vreg<256xui8>
          %30 = pto.vcvt %low_17, %2 {part = "P1", sat = "NOSAT"} : !pto.vreg<64xsi32>, !pto.mask<b32> -> !pto.vreg<256xui8>
          %31 = pto.vcvt %high_16, %2 {part = "P2", sat = "NOSAT"} : !pto.vreg<64xsi32>, !pto.mask<b32> -> !pto.vreg<256xui8>
          %32 = pto.vcvt %high_18, %2 {part = "P3", sat = "NOSAT"} : !pto.vreg<64xsi32>, !pto.mask<b32> -> !pto.vreg<256xui8>
          %33 = pto.vor %29, %30, %6 : !pto.vreg<256xui8>, !pto.vreg<256xui8>, !pto.mask<b8> -> !pto.vreg<256xui8>
          %34 = pto.vor %33, %31, %6 : !pto.vreg<256xui8>, !pto.vreg<256xui8>, !pto.mask<b8> -> !pto.vreg<256xui8>
          %35 = pto.vor %34, %32, %6 : !pto.vreg<256xui8>, !pto.vreg<256xui8>, !pto.mask<b8> -> !pto.vreg<256xui8>
          %36 = pto.dhistv2 %arg3, %35, %8, %c0_i32 : !pto.vreg<128xui16>, !pto.vreg<256xui8>, !pto.mask<b8>, i32 -> !pto.vreg<128xui16>
          %37 = pto.dhistv2 %arg4, %35, %8, %c1_i32 : !pto.vreg<128xui16>, !pto.vreg<256xui8>, !pto.mask<b8>, i32 -> !pto.vreg<128xui16>
          scf.yield %36, %37 : !pto.vreg<128xui16>, !pto.vreg<128xui16>
        }
        %result = pto.vlds %0[%c192] : !pto.ptr<si32, ub> -> !pto.vreg<64xsi32>
        %low, %high = pto.vdintlv %result, %result : !pto.vreg<64xsi32>, !pto.vreg<64xsi32> -> !pto.vreg<64xsi32>, !pto.vreg<64xsi32>
        %low_0, %high_1 = pto.vdintlv %low, %low : !pto.vreg<64xsi32>, !pto.vreg<64xsi32> -> !pto.vreg<64xsi32>, !pto.vreg<64xsi32>
        %low_2, %high_3 = pto.vdintlv %high, %high : !pto.vreg<64xsi32>, !pto.vreg<64xsi32> -> !pto.vreg<64xsi32>, !pto.vreg<64xsi32>
        %10 = pto.vcvt %low_0, %2 {part = "P0", sat = "NOSAT"} : !pto.vreg<64xsi32>, !pto.mask<b32> -> !pto.vreg<256xui8>
        %11 = pto.vcvt %low_2, %2 {part = "P1", sat = "NOSAT"} : !pto.vreg<64xsi32>, !pto.mask<b32> -> !pto.vreg<256xui8>
        %12 = pto.vcvt %high_1, %2 {part = "P2", sat = "NOSAT"} : !pto.vreg<64xsi32>, !pto.mask<b32> -> !pto.vreg<256xui8>
        %13 = pto.vcvt %high_3, %2 {part = "P3", sat = "NOSAT"} : !pto.vreg<64xsi32>, !pto.mask<b32> -> !pto.vreg<256xui8>
        %14 = pto.vor %10, %11, %6 : !pto.vreg<256xui8>, !pto.vreg<256xui8>, !pto.mask<b8> -> !pto.vreg<256xui8>
        %15 = pto.vor %14, %12, %6 : !pto.vreg<256xui8>, !pto.vreg<256xui8>, !pto.mask<b8> -> !pto.vreg<256xui8>
        %16 = pto.vor %15, %13, %6 : !pto.vreg<256xui8>, !pto.vreg<256xui8>, !pto.mask<b8> -> !pto.vreg<256xui8>
        %17 = pto.dhistv2 %9#0, %16, %8, %c0_i32 : !pto.vreg<128xui16>, !pto.vreg<256xui8>, !pto.mask<b8>, i32 -> !pto.vreg<128xui16>
        %18 = pto.dhistv2 %9#1, %16, %8, %c1_i32 : !pto.vreg<128xui16>, !pto.vreg<256xui8>, !pto.mask<b8>, i32 -> !pto.vreg<128xui16>
        %19 = pto.vcvt %17, %4 {part = "EVEN"} : !pto.vreg<128xui16>, !pto.mask<b16> -> !pto.vreg<64xui32>
        %20 = pto.vcvt %18, %4 {part = "EVEN"} : !pto.vreg<128xui16>, !pto.mask<b16> -> !pto.vreg<64xui32>
        %21 = pto.vcvt %17, %4 {part = "ODD"} : !pto.vreg<128xui16>, !pto.mask<b16> -> !pto.vreg<64xui32>
        %22 = pto.vcvt %18, %4 {part = "ODD"} : !pto.vreg<128xui16>, !pto.mask<b16> -> !pto.vreg<64xui32>
        %low_4, %high_5 = pto.vldsx2 %1[%c0], "DINTLV_B32" : !pto.ptr<ui32, ub>, index -> !pto.vreg<64xui32>, !pto.vreg<64xui32>
        %low_6, %high_7 = pto.vldsx2 %1[%c128], "DINTLV_B32" : !pto.ptr<ui32, ub>, index -> !pto.vreg<64xui32>, !pto.vreg<64xui32>
        %23 = pto.vadd %19, %low_4, %2 : !pto.vreg<64xui32>, !pto.vreg<64xui32>, !pto.mask<b32> -> !pto.vreg<64xui32>
        %24 = pto.vadd %20, %low_6, %2 : !pto.vreg<64xui32>, !pto.vreg<64xui32>, !pto.mask<b32> -> !pto.vreg<64xui32>
        %25 = pto.vadd %21, %high_5, %2 : !pto.vreg<64xui32>, !pto.vreg<64xui32>, !pto.mask<b32> -> !pto.vreg<64xui32>
        %26 = pto.vadd %22, %high_7, %2 : !pto.vreg<64xui32>, !pto.vreg<64xui32>, !pto.mask<b32> -> !pto.vreg<64xui32>
        %low_8, %high_9 = pto.vintlv %23, %25 : !pto.vreg<64xui32>, !pto.vreg<64xui32> -> !pto.vreg<64xui32>, !pto.vreg<64xui32>
        %low_10, %high_11 = pto.vintlv %24, %26 : !pto.vreg<64xui32>, !pto.vreg<64xui32> -> !pto.vreg<64xui32>, !pto.vreg<64xui32>
        pto.vsts %low_8, %1[%c0], %2 : !pto.vreg<64xui32>, !pto.ptr<ui32, ub>, !pto.mask<b32>
        pto.vsts %high_9, %1[%c64], %2 : !pto.vreg<64xui32>, !pto.ptr<ui32, ub>, !pto.mask<b32>
        pto.vsts %low_10, %1[%c128], %2 : !pto.vreg<64xui32>, !pto.ptr<ui32, ub>, !pto.mask<b32>
        pto.vsts %high_11, %1[%c192], %2 : !pto.vreg<64xui32>, !pto.ptr<ui32, ub>, !pto.mask<b32>
      }
      pto.set_flag[<PIPE_V>, <PIPE_MTE3>, <EVENT_ID0>]
      pto.wait_flag[<PIPE_V>, <PIPE_MTE3>, <EVENT_ID0>]
      pto.copy_ubuf_to_gm %1, %arg0, %c0_i64, %c1_i64, %c1024_i64, %c0_i64, %c1024_i64, %c1024_i64 : !pto.ptr<ui32, ub>, !pto.ptr<ui32, gm>, i64, i64, i64, i64, i64, i64
      return
    }
  }
}

Expected performance

CCE writes the 64 ui8 result directly to quarter 0 with a single instruction and lets dhistv2 read it under a VL64 predicate:

vlds(64×si32) → vcvt(si32→ui8, PART_P0)   # 1 instruction: 64 si32 → 64 ui8 @ quarter 0
            → dhistv2(..., mask=VL64)       # reads quarter 0 only

Actual performance

VMI's pto.vmi.vcvt(src, to_dtype=…) exposes no part argument, so the lowering cannot express "fill only P0." For a 4:1 narrowing feeding a nominally-256-element consumer, it instead deinterleaves the 64 si32 into sub-streams, casts each into a different quarter, and vor-merges into a full 256×ui8 register:

vlds(64×si32) → vdintlv ×3 (split into 4 sub-streams)
             → vcvt(part=P0) + vcvt(part=P1) + vcvt(part=P2) + vcvt(part=P3)   # 4 casts
             → vor ×3 (merge → 256×ui8)                                          # 3 merges
             → dhistv2(..., mask=VL64)                                          # reads quarter 0 only → P1/P2/P3 + merges wasted

Because dhistv2 is predicate-masked to VL64, only quarter 0 is ever read — the P1/P2/P3 casts and all three vor are dead work. This is the source of the +8 instructions per VLOOP iteration (5 → 13).

Profiling data (optional)

No response

Git commit

PTOAS 0.59