Skip to content

feat(vmi): VMIPredicateFold — DCE compile-time expert-pad vcmp/vsel - #581

Open
peanutchan wants to merge 1 commit into
mouliangyu:feature-vmifrom
peanutchan:feat/vmi-predicate-fold
Open

feat(vmi): VMIPredicateFold — DCE compile-time expert-pad vcmp/vsel#581
peanutchan wants to merge 1 commit into
mouliangyu:feature-vmifrom
peanutchan:feat/vmi-predicate-fold

Conversation

@peanutchan

Copy link
Copy Markdown
Collaborator

Summary

  • Adds VMIPredicateFold (-vmi-predicate-fold) to constant-prove VMI index ranges and fold identity expert-pad vcmp_lt+vsel before unified→legacy lowering.
  • Extracts shared VMIMaskUtils (isAllActiveSeed / isAllInactiveSeed) used by this pass and VMILowerUnifiedToLegacy.
  • Enables frontends (e.g. TileLang topk_gate) to always emit pad masking and drop hand-coded need_pad when num_experts is compile-time known.

Closes #580.

Motivation

Padding after VL-aligned vci / vadds is a common MoE pattern:

score = vsel(vcmp(index, num_experts, lt), score, -inf)

When the compare is provably all-true, this is dead work. Hand-gating with need_pad = (E != aligned_E) misses non-tail chunks and duplicates policy in every kernel.

Design

Item Choice
Pipeline slot VMI semantic pipeline, before VMILowerUnifiedToLegacy
Ranges vci, vadds(vci,C), vbrc(C), affine iv*stride+C + known scf.for bounds
Rewrites all-true/false vcmp → lattice; identity / constant vsel; pure-op DCE
A/B --disable-vmi-predicate-fold (PTO_FLAGS)

Lit: test/lit/vmi_new/vmi_predicate_fold_pad.pto.

Performance (camodel 950PR, N=4, pad always-on, fold OFF vs ON)

Config g RVECEX Δ elem/cycle Δ
E16 K2 (real pad, E≪VL) 1/2 0 0%
E64 K8 1 73→70 +1.0%
E64 K8 2 138→134 +1.2%
E384 K9 1 366→353 (−13) +1.0%
E384 K9 2 714→695 (−19) +1.2%

Identity-pad cases match the old hand-need_pad=False rooftop after fold; E≪VL correctly keeps pad.

Test plan

  • pto-test-opt test/lit/vmi_new/vmi_predicate_fold_pad.pto -vmi-predicate-fold | FileCheck
  • topk_gate camodel A/B with/without PTO_FLAGS=--disable-vmi-predicate-fold for E=16/64/384
  • Confirm dynamic num_experts does not fold (lit case)

Made with Cursor

Fold statically proven all-true/all-false vcmp into identity vsel and DCE
dead pad work so frontends can always emit expert-pad masking when E is
compile-time known (see mouliangyu#580).

Co-authored-by: Cursor <cursoragent@cursor.com>
@peanutchan

Copy link
Copy Markdown
Collaborator Author

Superseded for upstream by hw-native-sys#1151 against main (issue hw-native-sys#1150). This feature-vmi PR can stay for the fork branch if still useful.

@peanutchan

Copy link
Copy Markdown
Collaborator Author

Follow-up generalization (R1–R6 + R4 neutral reduce skip + camodel report) landed on main-based branch:

This feature-vmi PR (#581) remains the original pad-only fold; prefer hw-native-sys#1194 for the full optimizer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant