Skip to content

[Feature] pto.vmi.vcvt(fp16->bf16) is not supported which takes bad workaround solution #1335

Description

@sundyCoder

Summary

pto.vmi.vcvt(fp16->bf16) is not supported, which takes bad workaround solutions.

Cast path (still required by VMI/ptoas): si8 → si32 → f32 → f16 → f32 → bf16
(no direct f16→bf16; sitofp needs i32)

CCE example:

    __VEC_SCOPE__
    {
        vector_u8 vKvData;
        vector_f16 vCastFp16Res;
        vector_bf16 vOffset, vScale, vRes;

        vector_bool kvTypeMaskAll = pset_b8(PAT_ALL);
        vector_bool qTypeMaskLower64 = pset_b16(PAT_VL64);
        vector_bool qTypeMaskAll = pset_b16(PAT_ALL);
        vector_bool qTypeMaskHigher64;
        pxor(qTypeMaskHigher64, qTypeMaskLower64, qTypeMaskAll, qTypeMaskAll);

        // Load offset and scale (128 bf16 elements each)
        vlds(vOffset, offsetPtr, 0, UNPK_B16);
        vlds(vScale, scalePtr, 0, UNPK_B16);

        for (uint16_t i = 0; i < loopCnt; ++i) {
            // LoadAlign<UNPK_B8>: load 256 int8, unpack
            vlds(vKvData, srcPtr, 0, UNPK_B8);

            // Cast<half, int8, PART_EVEN>: int8→f16
            vcvt(vCastFp16Res, vKvData, kvTypeMaskAll, PART_EVEN, MODE_ZEROING);

            // Cast<bf16, half>: f16→bf16
            vcvt(vRes, vCastFp16Res, qTypeMaskAll, ROUND_R, MODE_ZEROING);

            // Add offset (lower 128 mask)
            vadd(vRes, vRes, vOffset, qTypeMaskAll, MODE_ZEROING);

            // Mul scale (lower 128 mask)
            vmul(vRes, vRes, vScale, qTypeMaskAll, MODE_ZEROING);

            // StoreAlign lower 64 elements
            vsts(vRes, dstPtr, i * VF_LEN_B16, NORM_B16, qTypeMaskLower64);

            // StoreAlign higher 64 elements (to second output buffer)
            // Note: ASC uses DATA_BLOCK_COPY with blockStride/repeatStride for interleaved store.
            // In CCE, we use a second store at offset + 64.
            vsts(vRes, dstPtr, i * VF_LEN_B16 + 64, NORM_B16, qTypeMaskHigher64);
        }
    }

ptoas 0.59 VMI solution:

    mask128 = pto.vmi.create_mask(VF_LEN, size=VF_LEN)
    v_offset = pto.vmi.vload(off_ub, 0, size=VF_LEN)
    v_scale = pto.vmi.vload(sc_ub, 0, size=VF_LEN)

    # One loop like CCE: 128 bf16 outputs per iteration (was 2×64).
    for i in range(LOOP_CNT):
        # Contiguous first-128 of each 256-byte block (VMI has no PART_EVEN 256128).
        v_kv = pto.vmi.vload(src_ub, i * 256, size=VF_LEN)
        # CCE: u8f16bf16. VMI: sitofp needs i32; f16bf16 needs f32 bridge.
        v_i32 = pto.vmi.vcvt(v_kv, to_dtype=pto.si32)
        v_f32 = pto.vmi.vcvt(v_i32, to_dtype=pto.f32)
        v_f16 = pto.vmi.vcvt(v_f32, to_dtype=pto.f16, rounding="R", saturate="SAT")
        v_f32 = pto.vmi.vcvt(v_f16, to_dtype=pto.f32)
        v_res = pto.vmi.vcvt(v_f32, to_dtype=pto.bf16, rounding="R", saturate="SAT")
        v_res = pto.vmi.vadd(v_res, v_offset, mask128)
        v_res = pto.vmi.vmul(v_res, v_scale, mask128)
        pto.vmi.vstore(v_res, dst_ub, i * VF_LEN, mask128)

Motivation / use case

Cast path (still required by VMI/ptoas): si8 → si32 → f32 → f16 → f32 → bf16
(no direct f16→bf16; sitofp needs i32)

Proposed API / behavior

No response

Alternatives considered

No response

Additional context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions