Skip to content

[Feature] VMI has no fused vmulscvt → Fa2VsstbNzStore 1.25× slower, +2× stores vs CCE #595

Description

@jyc-coding

Summary

Fa2VsstbNzStoreKernel (Flash-Attention-4 softmax output store: fp32→fp16 + deinterleave + NZ-format block-strided store) is a 1:1 CCE↔DSL port whose golden matches exactly, but the DSL lowering is instruction-heavy because VMI cannot express the fused multiply+cast. Three gaps, the first dominant:

  1. No fused vmulscvt in VMI (dominant) — CCE fuses scalar-mul + fp32→f16 cast into one vmulscvt(PART_EVEN). VMI must emit vmuls + vcvt separately, then repack the cast result with vbitcast + vpack LOWER + vbitcast (3 extra ops) before it is storable. Per stream: CCE 1 instr → DSL 5.
  2. Store doubling (no vdintlv stream-pairing) — CCE pairs two 64-f16 streams into one 128-f16 via vdintlv → one 128-elem vsstb. The DSL stores each stream separately → 4 vsstb/iter vs CCE's 2.
  3. PTOAS vstore repeat_stride encoding bug (separate, documented)vstore(..., repeat_stride=1) lowers to vsstb with Sm_rs = repeat_stride + 2 (0x00410002 instead of 0x00410001), forcing the low-level vsstb + explicit scalar bs path.

Motivation / use case

FA

Proposed API / behavior

No response

Alternatives considered

No response

Additional context

No response

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions