improve VMI FP4 dequant support - #1445
Draft
learning-chip wants to merge 2 commits into
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Minor fixes on top of #1443
Summary: keep the FP4 fix narrow and preserve existing VMI layouts
This branch starts at the PR1443 parent
209ac1ad8and keeps only thevalidated part of
578dec5de: the scratch-free FP4 conversion contract.The important change is scope. Preferred layout facts are now specific to
f4E2M1x2 → bf16x2; they are not a generic “8-bit to 32-bit” rule that cansilently change E4M3 or integer consumers. Physical FP4 conversion parts are
derived from the source layout, so contiguous lane-stride 4 uses P0, lane-
stride 2 uses P0/P2, and explicit packed/deinterleaved forms retain their
supported part mapping.
The lowering preserves the useful logical operation:
FP32 remains FP4→BF16 followed by BF16→FP32. The global
VMILoadOpnatural-layout override from PR1443 is deliberately absent, so producer preferences do
not override channel split/merge or other consumer constraints. Direct E2B
scale broadcasts remain available when their native layout is consumable.
The full Python 3.11 PTOAS build passed 1,846 of 1,847 discovered lit tests;
the only non-pass is the existing unsupported platform test. The focused
regressions covering FP4 lane-stride mappings, BF16-pair conversion, group
broadcast behavior, and channel split/merge all pass.
The remaining cast-back slowdown is therefore not an unresolved FP4 value
layout bug. It is the separate noncanonical packed-scale problem: the VMI
compiler needs a typed vector extraction/materialization contract that can
consume packed UE8M0 bytes without scalarizing them or requiring host-padded
scale slots. The next PTOAS step is to add that contract, its layout support
facts, and lit coverage before re-running the complete device matrix.