Skip to content

improve VMI FP4 dequant support - #1445

Draft
learning-chip wants to merge 2 commits into
hw-native-sys:mainfrom
learning-chip:zjw/castback_unify_0902
Draft

improve VMI FP4 dequant support#1445
learning-chip wants to merge 2 commits into
hw-native-sys:mainfrom
learning-chip:zjw/castback_unify_0902

Conversation

@learning-chip

@learning-chip learning-chip commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Minor fixes on top of #1443

Summary: keep the FP4 fix narrow and preserve existing VMI layouts

This branch starts at the PR1443 parent 209ac1ad8 and keeps only the
validated part of 578dec5de: the scratch-free FP4 conversion contract.

The important change is scope. Preferred layout facts are now specific to
f4E2M1x2 → bf16x2; they are not a generic “8-bit to 32-bit” rule that can
silently change E4M3 or integer consumers. Physical FP4 conversion parts are
derived from the source layout, so contiguous lane-stride 4 uses P0, lane-
stride 2 uses P0/P2, and explicit packed/deinterleaved forms retain their
supported part mapping.

The lowering preserves the useful logical operation:

vload(f4x2) → vcvt(bf16x2) → vinterpret_cast(bf16)

FP32 remains FP4→BF16 followed by BF16→FP32. The global VMILoadOp natural-
layout override from PR1443 is deliberately absent, so producer preferences do
not override channel split/merge or other consumer constraints. Direct E2B
scale broadcasts remain available when their native layout is consumable.

The full Python 3.11 PTOAS build passed 1,846 of 1,847 discovered lit tests;
the only non-pass is the existing unsupported platform test. The focused
regressions covering FP4 lane-stride mappings, BF16-pair conversion, group
broadcast behavior, and channel split/merge all pass.

The remaining cast-back slowdown is therefore not an unresolved FP4 value
layout bug. It is the separate noncanonical packed-scale problem: the VMI
compiler needs a typed vector extraction/materialization contract that can
consume packed UE8M0 bytes without scalarizing them or requiring host-padded
scale slots. The next PTOAS step is to add that contract, its layout support
facts, and lit coverage before re-running the complete device matrix.

@learning-chip learning-chip changed the title VMI FP4 quant support VMI FP4 dequant support Sep 2, 2026
@learning-chip learning-chip changed the title VMI FP4 dequant support improve VMI FP4 dequant support Sep 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants