Feature/vpto cpp bridge generalization - #1326
Closed
kuri780 wants to merge 49 commits into
Closed
Conversation
VPTO emits LLVM IR, which cannot instantiate the C++ templates that
implement PTO-ISA TPush/TPop/TFree. Reuse the templates instead of
reimplementing the FIFO protocol: lower the internal pipe ops to a fixed
C ABI wrapper (vpto_bridge.cpp), compile the wrapper to per-target
bitcode with Bisheng, and merge it into the VPTO device LLVM module via
llvm-link before device object compilation.
- CANN900 emitter: lower initialize_l2l_pipe/tpush/tpop/tfree to
pto_vpto_pipe_* calls; PipeType becomes an opaque pointer, TileBufType
an i64 address; TPipe storage lives on the kernel stack and
pipe_finish runs before every return; TPOP rebinds the declared tile
through the returned FIFO slot address
- VPTOSplitCVModule: split modules by pto.kernel_kind function
attributes, pruning functions of the other kind
- PTO verifier: treat modules with a kernel_kind attribute as inside a
kernel
- FoldTileBufIntrinsics: keep declare_tile rebound by tpop unfolded so
the runtime slot address is not replaced by the placeholder
- ObjectEmission: link PTOAS_VPTO_{CUBE,VECTOR}_BRIDGE_BITCODE into the
device module with the Bisheng llvm-link before compilation
- TileOpExpansionUtils: exclude internal pipe ops from TileLib
expansion
Co-Authored-By: Claude <noreply@anthropic.com>
Add a VPTO backend case that proves the TPOP-returned FIFO slot address
reaches downstream vector instructions: the output has no Cube-to-GM
bypass and is written only from the Vec tile rebound by TPOP, so a
golden compare of the full 128 f32 elements proves the data really
flowed through the FIFO.
Extend run_host_vpto_validation.sh to build the per-target bridge
bitcode for cases with vpto_bridge.cpp and pass it to ptoas through
PTOAS_VPTO_{CUBE,VECTOR}_BRIDGE_BITCODE.
Co-Authored-By: Claude <noreply@anthropic.com>
Design record for reusing PTO-ISA C++ templates from the VPTO backend: wrapper ABI, TPipe storage lifecycle, bitcode merge via llvm-link, CA simulator validation results, limitations, and next steps. Co-Authored-By: Claude <noreply@anthropic.com>
Per-file change summary for the branch: what changed, why, how the changes depend on each other, and the end-to-end walk-through of how TPush/TPop is implemented from kernel.pto to the CA simulator run. Co-Authored-By: Claude <noreply@anthropic.com>
fifo-tile-data-consume writes the internal pipe ops directly (initialize_l2l_pipe/tpush/tpop/tfree with explicit attributes); it does not go through the public-op frontend lowering. The frontend PTOLowerFrontendPipeOpsPass and the internal ops themselves are pre-existing on main and are not changed by this branch. Co-Authored-By: Claude <noreply@anthropic.com>
Replace the conservative 256-byte alloca for the TPipe bridge storage with a call to pto_vpto_pipe_size() exported by the wrapper, so the exact sizeof(TPipe) from the instantiated PTO-ISA template is used at runtime. LLVM alloca supports a dynamic size; alignment stays static 8 (LLVM requires a constant alignment, and alignof(TPipe) <= 8 for the A5 layout). The fifo-tile-data-consume case now provides pto_vpto_pipe_size() in both the cube and vector bridge bitcode, and the case still passes end to end on the CA simulator. Co-Authored-By: Claude <noreply@anthropic.com>
A simulator experiment (no-op finish) showed single-launch FIFO data correctness does not depend on the TPipe destructor handshake: the consumer-side TPipe constructor pre-sets the freed flags, so the handshake never blocks in the fixed specialization. Remove pto_vpto_pipe_finish from the bridge ABI and the emitter's per-return insertion, and drop the explicit PIPE_V wait in pop (TPOP performs the wait internally). Multi-round push / cross-launch FIFO reuse is out of scope; reintroduce finish if such cases appear. Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
…ing (Phase 0) Split the PoC pipe bridge into a layered, reusable bridge machinery implemented from scratch: - Add generic bridge ops pto.bridge_call / pto.bridge_inttoptr - Add the TPipe family pass: internal pipe ops and tile handles become bridge ops; config validation, storage handle flow, and TPOP runtime rebinding are resolved here - Add the generic bridge lowering pass: whitelist-driven, family-agnostic conversion of bridge ops into wrapper calls plus the stateful-object storage pattern (size query + stack alloca) - Add the YAML whitelist reader and the fifo-tile-data-consume whitelist - Wire both passes into the VPTO pipelines - Record the generalization plan in docs/designs Co-Authored-By: Claude <noreply@anthropic.com>
Finalize the from-scratch bridge machinery and validate it end to end (CA simulator, 128-element compare passed): - Family pass: RAUW the storage handle and the materialized pointer when rewriting init/tile_buf_addr so erased ops leave no dangling uses - Generic bridge lowering: mark unknown ops dynamically legal so the conversion driver accepts the calls/alloca/declarations the patterns create; declare LLVM/func dependent dialects; fix enumerate binding order, copyOptionValuesFrom argument, StringSet includes, and the generated BridgeCallOp builder signature (results-first) - Wire the generic pass into the CANN900 runPipeline (the Beta1 pipeline had it; the A5 test path goes through CANN900) - Delete the PoC-era pipe bridge patterns from the VPTO emitter; pipe family handling now lives solely in the new passes - Restore lit tool deps dropped by the LLVM 19 cherry-pick that refer to the not-yet-ported VPTOScheduler - Inject PTOAS_VPTO_BRIDGE_WHITELIST from the host validation script Co-Authored-By: Claude <noreply@anthropic.com>
…eration (Phase 2)
- Family pass: drop the hardcoded slot_size/slot_num/flag_base validation;
pipe attributes flow verbatim into the collected bridge specialization.
- BridgeTokenUtils: render TPipe/Tile/TileSplit C++ template tokens from the
collected spec (element type mapping included).
- New module pass pto-emit-vpto-bridge-wrapper: merges the per-function
pto.vpto.bridge.func_spec attributes into the module spec (concurrent func
pass instances must not write the shared module attribute), rejects
conflicting specializations, and renders pto.vpto.bridge.wrapper_source
with __DAV_CUBE__/__DAV_VEC__ guarded entries.
- ObjectEmission compiles the generated wrapper source into both cube and
vec fatobj halves; the hand-written vpto_bridge.cpp and the
PTOAS_VPTO_*_BRIDGE_BITCODE env channels are removed
(PTOAS_VPTO_BRIDGE_WHITELIST stays).
- Whitelist tmpl_map rows ({source, field, target}) are schema-validated;
pipe-family sources are restricted to pipe.init/tile.
- lit: config matrix (variant slot_size/slot_num/flag_base/nosplit, f16,
acc/vec tiles), C2V/V2C wrapper source checks, tmpl_map and cross-function
conflict diagnostics (5 tests + 3 fixtures).
- Validated: full lit suite 1767/1772 (4 pre-existing failures), simulator
fifo regression and a slot_num=4/flag_base=8 variant both compare passed.
…wrapper gen (Phase 3) Add the MATMUL family as the second bridged interface on top of the Phase 2 infrastructure, with zero changes to the family-agnostic VPTOBridgeLowering pass and the ObjectEmission channel: - PTOLowerMatmulFamilyOps: lowers pto.tmatmul / pto.tmatmul.acc to pto.bridge_call with planned-address ABI values (3x / 4x i64), reusing the whitelist-driven routing. - VPTOBridgeWrapperGen: renders matmul-family wrappers (__DAV_CUBE__ side), including the acc_phase template argument merged per module. - VPTOBridgeWhitelist: matmul entries with tmpl_map source validation (left_tile/right_tile/result_tile/acc_in_tile). Tests: - lit: family lowering spec/wrapper checks, unsupported-tile and unknown-source whitelist diagnostics, matmul whitelist fixtures. - simulator e2e: kernels/cube-matmul-bridge 16x16x16 f16 matmul routed through the generated TMATMUL wrapper, compare passes on DEVICE=SIM. Full lit regression: only the 4 pre-existing failures remain.
…_map coverage (Phase 4)
…agnostics The scf.for consumption probe confirms the bridge rebinds the consumer tile per iteration: the pop bridge_call result stays a loop-body SSA value feeding bridge_inttoptr, matching the EmitC reference loop shape. Pin this with vpto_bridge_pipe_loop_consume.pto (BRIDGE/EMITC prefixes). Two sequential TPOPs on the same declared tile previously crashed in DictionaryAttr::get on duplicated spec keys (and would otherwise overwrite the tile's single recorded slot address). Reject the second pop with an explicit diagnostic and add a duplicate-spec-key fallback check before writing the function spec. Diagnostic pinned by vpto_bridge_pop_rebind_diag.pto.
Restore pto-vpto-scheduler-tracker-test to the lit tool list (its removal only served the dropped LLVM 19 downgrade cherry-pick), and pin the function-level pto.kernel_kind attributes that the bridge split deliberately stamps onto cloned functions in the two mainline split tests.
Drop the three PoC-era design docs (they describe the predecessor branch, not this generalization), remove the fifo-variant-config end-to-end case whose compiler-side coverage already lives in the spec config matrix lit test, and restore test/lit/CMakeLists.txt and lit.cfg.py to their mainline content now that the dropped LLVM 19 downgrade cherry-pick no longer justifies their divergence.
Keep docs/designs/vpto-cpp-bridge-generalization.md out of the change set (the PR carries no design docs) and remove the dangling references to it from the bridge code comments and whitelist fixtures.
…amily pass
- Extract the shared pure-mapping token layer PTOCppTokens.{h,cpp}
(IR facts -> C++ spellings, qualifier parameter); the EmitC and
bridge token builders become thin wrappers, parity stays zero-diff
- Extend the whitelist schema: lowering (declarative|family), abi rows
binding operand position/arg label/template role, tmpl_map attr rows
with enum_type/omit_value; replace the hardcoded matmul tmpl source
set with structural validation for declarative entries
- Add PTOLowerDeclarativeBridgeOps (func-level nested pass) lowering
whitelist-routed ops from the whitelist description alone; enum
attributes reflect through the new PTO_EnumTokenAttr interface
(accPhase implements it); attr lookup tries the snake_case spec key
and its camelCase ODS spelling
- Drop PTOLowerMatmulFamilyOps; the pipeline mounts the declarative
pass before the pipe family pass; unrouted matmul ops keep the mad
expansion fallback
- Migrate matmul fixtures/tests to the declarative schema (positive
CHECKs unchanged, byte-identical output), refresh diagnostic CHECKs,
add declarative_binding_diag and declarative_unrouted_passthrough
lits; full regression 1839/1840 (1 unsupported), DEVICE=SIM e2e
cube-matmul-bridge and fifo-tile-data-consume compare passed
Fuse the print-via-cce-wrapper IR layer onto the bridge-generalization branch without rebasing. This first commit lands the dialect-level pieces that every downstream consumer (EmitC backend, VPTO emitter, wrapper TU) shares: - Add PrintEncoding.h: the format-string state machine (analyzePrintFormat) classifying the single conversion specifier into Float / SignedInt / UnsignedInt. Header-only, consumed by the verifier here and by the VPTO emitter lowering in a follow-up commit. - Enable PrintOp::verify(): restrict the scalar operand to the supported print set (f16/bf16/f32/f64, i8/i16/i32/i64) and validate that the format string carries exactly one conversion specifier whose kind matches the operand type. - Exclude TPrintOp from TileLib template expansion via isTileLibExpandableOp: the mainline collapsed the per-pass skip lists into this single predicate, so the debug op is exempted here instead of in each walker. Lit: new print_scalar_emitc (verifier accepts + EmitC lowers to cce::printf) and print_scalar_invalid (six diagnostic scenarios via split-file). Existing print/tprint cases and the 23 bridge lits stay green.
Port the print-via-cce-wrapper lowering into the VPTO emitter: - pre-scan creates _ptoas_printf_fmt_N string globals (addrspace 1) before dialect conversion (DenseElementsAttr values for LLVM 19 ModuleTranslation); - addDTDataParamToEntryFunctions declares the six pto_print_* shims and appends the hidden ptr addrspace(1) DTData parameter to every entry; - LowerPrintOpPattern / LowerTPrintOpPattern emit typed wrapper calls (f16/bf16 fpext, f64 fptrunc, narrow ints sext/zext to i64); - injectPrintPrologue adds pto_print_init at entry and pto_print_finish before each return after conversion; - LowerAllocTileOpPattern maps alloc_tile to a null ptr6 UB placeholder for non-expandable consumers (tprint). Covered by test/lit/vpto/print_scalar_vpto_llvm.pto and tprint_vec_vpto_llvm.pto; full vpto suite (566) green at -j8.
…ode host stub
Wire the print path into object emission, reusing the bridge channel's
bitcode-merge machinery:
- linkDeviceLLVMBitcode now takes multiple extra bitcode inputs so the
kernel IR, the bridge wrapper bc and the print wrapper bc are merged
in a single llvm-link invocation.
- emitVPTO{Vector,Cube}DeviceObject compile tools/ptoas/cce/pt_print.cpp
with bisheng driver mode (--cce-enable-print, no -cce-enable-mix) per
target CPU and add the resulting bitcode to the link inputs when the
module uses pto.print / pto.tprint.
- Host stub compilation switches to bisheng driver mode under usesPrint
(DebugTunnel pulls libstdc++ headers CC1 cannot resolve); the fatobj
is then extended with pt_print_host.cpp via ld.lld -r to override
__DebugTunnel_Close for post-sync buffer flush.
- ptoas.cpp detects PrintOp/TPrintOp before lowering and plumbs
PTOASCompileResult.usesPrint through the driver into emitFatobjLLVM;
the runtime/standalone-stub paths keep usesPrint=false.
Design notes and verification summary for the declarative + family channel bridge architecture that the print integration builds upon.
…driver-mode host stub" This reverts commit d7bd6e6.
… emission" This reverts commit 4ada3d6.
…IR layer)" This reverts commit 5652657.
- lowering 缺省值翻转为 declarative,退出通道改名 custom(消除与 wrapper 字段的撞名);漏标的 custom 条目在解析期即被拒并点名缺失标注 - 白名单 family 字段改名 wrapper:wrapper 是生成单位(同名条目共享一份 翻译单元),wrapper 生成器改为按使用中条目的 wrapper 字段选择渲染器, 不再嗅探 spec key;混用诊断由实际 wrapper 集合自描述,无渲染器的值 报明确诊断并列出可用集合 - op: internal 条目豁免通道校验;残留路由检查按通道点名责任方 - 修复 declarative_binding_diag 既有 flake:改用 -split-input-file, 四个场景各自独立 pass-manager 运行(此前诊断条数在 1~4 间随机) - 新增 default_channel_diag 与 unknown_wrapper_diag 回归测试 - 同步设计文档:白名单 schema、分派机制、能力矩阵
…rappers section - whitelist schema: top-level wrappers section (name/includes/core) plus per-entry call/tmpl_args; entry and abi type default for declarative entries; tile typedefs derive from abi roles (tile tmpl_map rows rejected) - WrapperGen: dispatch on entry channel; declarative wrappers render via the generic renderer, custom wrappers keep dedicated renderers (pipe) - drop the matmul renderer, BridgeMatmulSpec and the matmul entry spec key constants; built-in matmul entries migrated to call/tmpl_args
- matmul fixtures rewritten onto the wrappers section with call/ tmpl_args and role-only tmpl_map; checkpoints follow the generic renderer output (role typedefs in target-name order) and the new diagnostics - new: vpto_bridge_declarative_wrapper_source.pto pins the migrated matmul wrapper_source byte-for-byte against the old hardcoded renderer; vpto_bridge_whitelist_render_schema_diag.pto covers the new schema edges (missing call / unbound tmpl_args / bad core / missing wrappers section)
Schema example shows the wrappers section, call/tmpl_args and the role-derived typedefs; dispatch and rendering sections describe the generic renderer split; capability and test tables gain the declarative_wrapper_source / render_schema_diag / tadd coverage.
Built-in whitelist gains the vec_elem wrapper (TAdd include, vec core guard) and a pto.tadd entry with op/wrapper/call/abi; the generic declarative renderer emits the pto_vpto_add entry (derived name, no template arguments) under __DAV_VEC__. No pass or generator code knows about tadd. Lit covers the zero-env declarative lowering (vec tile spec tokens, bridge_call) and the rendered wrapper source.
Vector-side elementwise add routed through the declarative bridge: the built-in vec_elem whitelist registration renders the pto_vpto_add entry (pto::TADD under __DAV_VEC__) with zero C++ knowing about tadd. The kernel moves two 8x16 f32 tiles into planned UB allocations via mte_gm_ub, adds them through the bridge wrapper and stores the result with mte_ub_gm. DEVICE=SIM compare passed on all 128 elements.
…ntation - add an executive overview of the branch deliverables - add a usage section walking through whitelist-only onboarding of a mechanical op (tadd) with the rendered wrapper excerpt - restructure the lit coverage into the full 28-case matrix grouped by routing/channel, pipe family, declarative families and schema/spec validation; drop process narrative from the test notes - fix facts against the code (tpush entry spelling, typedef ordering description, dispatch wording) and remove transitional phrasing
…ck-reference appendix
Cut the process narrative (PoC limits, EmitC comparison tables) and keep only design outcomes. Switch the onboarding walkthrough to the minimal base-matmul registration riding all convention defaults, and replace the schema field appendix with a one-page derivable-fields-and-failure-points quick reference.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
概述
把此前 TPipe 专用、配置硬编码的 C++ 接口桥接 PoC,泛化为编译器内置的通用通道:
VPTO 无法用 LLVM IR 表达的 PTO-ISA C++ 模板接口(TPush/TPop/TFree、TMATMUL),
现在由 IR 属性驱动参数收集、YAML 白名单驱动路由、编译器自动生成 wrapper 源码并
编译为 device bitcode 合入 fatobj——
ptoas --pto-backend=vpto零环境变量开箱即用。设计要点
三层分离(约束:通用混编 pass 不感知任何家族语义):
PTOLowerPipeFamilyOps/PTOLowerMatmulFamilyOps):理解 op 语义,产出
pto.bridge_call(callee + ABI 值),家族语义(如 TPOP地址重绑定)经 SSA result 在本层完成;
VPTOBridgeLowering,全文无家族字样):按白名单abi字段校验并机械降级为
func.call+ 外部声明;白名单命中但未转换的 op 报明确诊断,不静默回退;
VPTOBridgeWrapperGen+ObjectEmission):家族 pass 收集的 spec 经模块级确定性合并,按白名单
tmpl_map行驱动渲染wrapper 源码(typedef +
__DAV_CUBE__/__DAV_VEC__守卫入口),cube/vec两路编译 bitcode 注入各自 fatobj。
白名单三级解析链:pass option →
PTOAS_VPTO_BRIDGE_WHITELISTenv → 内置默认。模板 token 构建(
VPTOBridgeTokens)参照 EmitC 规则独立实现,EmitC 侧零改动,由
vpto_bridge_emitc_token_parity.pto双路径比对防漂移。能力范围
f16、任意 tile 形状(含 NoneBox)、split≠1、循环内消费;
accPhase 模板实参、float8/float4 标量 token。
测试
fifo-tile-data-consume、cube-matmul-bridge均 compare passed。Reviewer 指引
VPTOBridgeLowering.cpp(358 行,家族无关的机械降级)入手,再看两个家族 pass 的差异——MATMUL 接入时通用 pass 与 ObjectEmission 零改动,
这是分层假设的核心证据;
单线程进行(
mergeFuncSpecsIntoModule),冲突报诊断;VPTOSplitCVModule新增函数级 kernel_kind 拆分(split 后函数携带
pto.kernel_kind属性,下游__DAV_CUBE__/__DAV_VEC__守卫依赖);
FoldTileBufIntrinsics保护被 TPOP 重绑定的 declare_tile。