How to develop vpto ops

[mouliangyu]

Now our ptoas only lower a5 codes to vpto backend. To support a2a3 ops, some inputs and major files to change as below:

1. Define ops in VPTOOps.td(or we need a new td file like VPTOA2A3Ops.td), the ops comes from cce builtin(see attached file).
2. Recommendation: Some CCE builtins have type suffixes, such as __builtin_cce_scatter_vadds_f16. It's recommended to define them directly as scatter_vadds and deduce the type based on the pointer type in the MLIR. For example, pto.scatter_vadds pto.ptr<f16, ub>, ...
3. Write the op constraints in VPTO.cpp(or VPTOA2A3.cpp)
4. Write the vpto->llvm converter in VPTOLLVMEmitter.cpp, see the llvm intrinsic signature in attached file
5. Write end-to-end testcases in test/vpto/cases/micro-op, a case is a directory contains main.cpp+launch.cpp+kernel.pto+golden.py+compare.py
6. Write micro-op manual in docs/isa/micro-isa

Questions:

1. Vector add builtin names __builtin_cce_vadd(ptr, ptr, ptr, ...), should we define it as pto.vadd? It conflicts with pto.vadd in a5.

   maybe we can define all a2a3 ops like pto.membase.vadd. and relative filename as VPTOMembase.td, VPTOMembase.cpp

2. Does the a2a3 fatobj packing same as a5?

The attached builtin list has been moved to docs/designes/a2a3-vector-builtins.md branch: feature-vpto-a2a3

[Zhendong404]

Another related question: how to design the corresponding PTODSL syntax for the A2A3 micro instructions?
I prefer the following coding style:

pto.vadd(TileOut[row, col:], TileIn1[row, col:], TileIn2[row, col:], mask)

And this style is also acceptable if the user need:

ptr_out = TileOut.as_ptr() + offset
ptr_in1 = TileIn1.as_ptr() + offset
ptr_in2 = TileIn2.as_ptr() + offset
pto.vadd(ptr_out, ptr_in1, ptr_in2, mask)

[learning-chip]

Is there a practical need to go below tile-level ops for A2A3? There is no custom Vector Function to fuse, so TADD is able to 100% replicate what vadd can do?

If the answer is no, then we don't need to expose vpto to A2A3 frontend, but only use it as backend for tile-level ops.

[mouliangyu]

actually a2a3 micro op has very complex operand list to determine how to loop

[Zhendong404]

Is there a practical need to go below tile-level ops for A2A3? There is no custom Vector Function to fuse, so TADD is able to 100% replicate what vadd can do?

If the answer is no, then we don't need to expose vpto to A2A3 frontend, but only use it as backend for tile-level ops.

So we don't have users who want to write A2A3 micro-instructions directly?

[WenboCodes]

We do. ByteDance wants A2A3 micro instructions for speeding up compilation time, for auto-tuning purpose. However, we can put A2A3 pto-DSL in a lower priority until we find a real use case.

[castigli]

I would prefer a separate filesVPTOA2A3Ops.td/cpp. Regarding the names, what about simply pto.a2a3.*?

Is there a practical need to go below tile-level ops for A2A3? There is no custom Vector Function to fuse, so TADD is able to 100% replicate what vadd can do?

I think the correct way is to lower/expand TADD in terms of a2a3 ops and then emit llvm IR

[WenboCodes]

could there be in the future different signature of the same pto.membase.vadd ? For consistency then the a5 op should be pto.simd.vadd or pto.reg.vadd?

You are correct. However, pto.vadd for A5 has already been released to customers. It's difficult to ask them to refact their code.

[WenboCodes]

could there be in the future different signature of the same pto.membase.vadd ? For consistency then the a5 op should be pto.simd.vadd or pto.reg.vadd?

To make it shorter, I feel pto.mem.vadd is better.

[castigli]

I see, I guess once things are more stable we could accept both forms and slowly deprecate the old one.
pto.membase.* seems still a bit ambiguous to me, what about pto.ub.vadd?
On what repo/branch this should be implemented?

[WenboCodes]

I see, I guess once things are more stable we could accept both forms and slowly deprecate the old one. pto.membase.* seems still a bit ambiguous to me, what about pto.ub.vadd? On what repo/branch this should be implemented?

Sounds ok to me. @mouliangyu which branch you recommend @castigli working on?

[mouliangyu]

@castigli @WenboCodes any branch is ok

[castigli]

In the document I see 2 signatures for some ops, one is the one that I am familiar with from CCE (i.e. with block and repeat strides) what is the other one with config param? For instance

void __builtin_cce_vadd(__ubuf__ int16_t* dst, __ubuf__ int16_t* src0, __ubuf__ int16_t* src1, uint64_t config);
void __builtin_cce_vadd(__ubuf__ int16_t* dst, __ubuf__ int16_t* src0, __ubuf__ int16_t* src1, uint8_t repeat, uint8_t dstBlockStride, uint8_t src0BlockStride, uint8_t src1BlockStride, uint8_t dstRepeatStride, uint8_t src0RepeatStride, uint8_t src1RepeatStride);

The document seems to suggest that the llvm intrisic maps to the the one with config

declare void @llvm.hivm.VADD.s16(ptr addrspace(6) nocapture writeonly, ptr addrspace(6) nocapture readonly, ptr addrspace(6) nocapture readonly, i64) #1

Is the config param packing the strides? If so how?

[castigli]

cce_aicore_intrinsics_3101.h has the info I was looking for

[castigli]

what is the intrinsic for set_vector_mask()? it is needed for some edge cases of pto.tadd -> pto.ub.vadd

[mouliangyu]

i pushed a branch feature-vpto-a2a3. you can see docs/designs/a2a3-vector-builtins.md