examples(swiglu_mx_quant): CCE + VMI - #547
Open
MirkoDeVita98 wants to merge 3 commits into
Open
Conversation
MirkoDeVita98
force-pushed
the
swiglu-mx-quant-vmi
branch
2 times, most recently
from
July 21, 2026 11:53
237b884 to
5592ecc
Compare
mouliangyu
force-pushed
the
feature-vmi
branch
3 times, most recently
from
July 23, 2026 09:32
84a34b5 to
04d95b2
Compare
added 3 commits
July 23, 2026 14:50
…omparison Add a kernel-test harness for swiglu_mx_quant with a CCE backend and a 1:1 PTODSL/VMI port (bf16, e4m3, OCP, swigluMode 0), plus the numpy golden and a byte-exact tiling-data port. The VMI port is byte-exact vs the golden at every shape. With double-buffering (per-tile MTE2/MTE3 DMA overlapped with vector compute, mirroring CCE), it is competitive with CCE end-to-end: faster up to 256x512, within ~6% at 512x1024, and ~11% at 1024x2048. Per-block MX scaling uses grouped vcmax(group=NBLK) + vbrc(group=NBLK); VEXP/VDIV op counts match CCE exactly. The residual gap at scale is CCE running the e8m0 scale/guard bit-math on the scalar pipe while VMI uses the vector pipe. SMX_VMI_FORCE=sb|db selects the buffering mode. Also drop two stray [[maybe_unused]] attributes on data members in PTOToEmitC.cpp that broke the source rebuild under GCC -Werror=attributes.
The e8m0 mxscale bytes are stored in UB by the existing PK_B16 vsts CONTIGUOUSLY (scaleBytesPerRow bytes per row, packed back-to-back). DmaUb2GmScale previously issued a single copy_ubuf_to_gm_align_v2 with nBurst=dim0Size; on this target only the first burst of a multi-burst UB->GM transfer lands, so every row after row 0 was written as zero. Result: scale_eq=100%% at 1 row/core (64x512) but 50%% at 2 rows/core (128x512, even rows ok / odd rows zero) and lower for more rows/core. Fix (scale DMA only, PK_B16 store unchanged): copy one row per nBurst=1 burst, advancing the UB source by scaleBytesPerRow and the GM dest by outputScaleRowBytes. Verified in cannsim (bf16_e4m3_rint_ocp): scale_match now 100%% at 64x512, 128x512, 256x512, 512x1024, 1024x2048 (was 100/50/25/.../.. ). y_match is unchanged by this scale-only change (e.g. 128x512 y=72.39%% before and after) and reflects a pre-existing vexp/vdiv precision limitation in the double-buffered compute path, not the scale DMA.
DmaUb2GmY had the identical dropped-burst bug as the scale DMA: a single copy_ubuf_to_gm_align_v2 with nBurst=dim0Size only lands the first burst on this target, so for >1 row/core it wrote only row 0 of each core and rows 1+ read back as zero. This (not vexp/vdiv precision) was the cause of the batched y_match gap (128x512 y=72%%, 256x512 y=59%%, 512x1024 y=56%%, 1024x2048 y=54%%). Fix (batched tiling preserved): issue one nBurst=1 burst per row, rowBytes valid bytes from the contiguous outUb row (src advances by the aligned UB row size dim1AlignSizeNow, or /2 for fp4), landing at the GM row (dst advances by the full GM row pitch halfInput, or dim1/4 for fp4). Verified in cannsim (bf16_e4m3_rint_ocp), BATCHED tiling, y AND scale: 64x512, 128x512, 256x512, 512x1024, 1024x2048 -> y=100.00%%, scale=100.00%% (all).
MirkoDeVita98
force-pushed
the
swiglu-mx-quant-vmi
branch
from
July 23, 2026 14:52
5592ecc to
99e111c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.


swiglu_mx_quant: CCE + VMI kernel-test, cannsim cycle comparison + CCE multi-row DMA fix
Summary
Adds a
swiglu_mx_quantkernel to the kernel-test framework with both a CCEreference and a 1:1 PTODSL/VMI port (bf16 in, e4m3, OCP scale, swigluMode 0),
plus a cannsim-based cycle comparison. Along the way it fixes a real multi-row
correctness bug in the CCE reference.
Both backends are byte-exact vs the golden at every swept shape. On cannsim
cycles the two are close, with a crossover: VMI is faster at small shapes, CCE
faster at large ones.
What's included
kernels/swiglu_mx_quant/):spec,reference(numpygolden — SwiGLU silu-gate + per-32-block MX-fp8 quant + a byte-exact port of the
CCE tiling calc),
runtime,tile_config,cce/+vmi/backends,cycle_metrics.vmi/backend.py, PTODSL): per-32-block MX scaling via groupedvcmax(group=NBLK)+vbrc(group=NBLK); e8m0/recip bit-math;vcvtf8e4m3;double-buffered (per-tile MTE2/MTE3 DMA overlapped with vector compute) for
shapes with ≥2 rows/core. VEXP/VDIV op counts match CCE exactly (equal work).
cce/, lifted from a5) + its bishengCMakeLists.CCE correctness fix
The CCE reference produced wrong output for shapes with >1 row per core
(
rows > 64): a core wrote only its first row; later rows read as zero (y 72%→54%,scale 50%→0% as rows/core grew). Root cause:
copy_ubuf_to_gm_align_v2drops allbut the first burst when
nBurst>1on this cannsim/bisheng target — hitting boththe scale and data DMAs. Fix:
DmaUb2GmScaleandDmaUb2GmYnow issue onenBurst=1burst per row (UB source advances contiguously, GM dest by the rowstride). DMA-only; compute and tiling unchanged; batched tiling preserved.
Result — cannsim,
bf16_e4m3_rint_ocp, all shapes now y = scale = 100%:Fair cycle comparison
Both backends byte-exact. Metric = total kernel instruction span (per-core,
blkDim=64) as the cannsim wall-clock proxy; fresh consistent sweep:
Crossover ~256–512: VMI wins on the small shapes (fixed per-tile overhead
dominates there and VMI's pipeline is tighter), CCE wins at scale (its vector-engine
work is very compact — e.g. RVEC span 3287 vs VMI 14115 at 1024×2048 — while VMI's
span grows with the wider MX-scale vector work).
Caveats
nBurst>1burst-drop is likely a cannsim limitation, not real hardware;on hardware the original single-multi-burst CCE DMA would work without the
per-row-burst workaround (which adds some MTE3 overhead), so CCE's true hardware
span could be lower. This comparison is cannsim-specific.
__simd_vf__/bishengpacks vector ops differently from VMI's VPTO lowering); total instruction span is
the fair cross-backend proxy.
swigluMode 1 are not yet exercised by the VMI backend.
Reproduce
Everything runs under cannsim . Commands from
test/kernel-test/.Env (CANN + ptoas source build):
Build the CCE reference (bisheng):
cmake -S kernels/swiglu_mx_quant/cce -B kernels/swiglu_mx_quant/cce/build \ -DASCEND_HOME_PATH="$ASCEND_HOME_PATH" -DASCEND_DRIVER_PATH="$ASCEND_DRIVER_PATH" cmake --build kernels/swiglu_mx_quant/cce/build --target swiglu_mx_quant_cceCorrectness (byte-exact y + scale vs the numpy golden):
Cases/shapes:
bf16_e4m3_rint_ocp_{64x512,128x512,256x512,512x1024,1024x2048}.Cycles + Perfetto trace: