Add Torch and Triton kernels for Torch2FlyDSL and Triton2FlyDSL translation#45
Conversation
|
Validation All 99 new tasks were run through the project's Task Validator checks (the 10 checks in agents/task_validator) on gfx942/MI300X: config schema, source files, target symbols, compile, correctness, performance, correctness-implementation review, self-containedness, hang check, and result-template compatibility. Result: 99/99 PASS (48 torch2flydsl + 51 triton2flydsl), 0 FAIL. The 11 FP4/MX gfx950-only tasks validate as a clean skip on gfx942 (exit 0) — the expected platform-support behavior; all other tasks pass compile + correctness (against the real aiter op / in-source reference at the tight gate) + performance. Per-task validation_report.yaml and the aggregate validation_summary.yaml are available in a separate commit, these should be removed from the PR once it is reviewed and approved. |
|
Hi, quick heads-up: we are currently working on a larger refactor #48 that moves the benchmark/runtime flow to a Docker-first environment. This touches a lot of shared infrastructure, so reviews and merges may be slower than usual for a bit. Once that refactor lands, I’ll help rebase open PRs onto that one. Sorry for the delay, and thanks for your patience. |
de5418d to
b069e2b
Compare
irvineoy
left a comment
irvineoy
left a comment
There was a problem hiding this comment.
Hi, first of all, thank you for your PR.
I rebased this branch on top of the latest main and fixed a few known issues in the task configs / validation setup.
I also ran the Validator Agent on all newly added tasks in this PR on gfx950/MI355X. Some tasks did not pass validation, and I collected the non-PASS results in the report below.
pr45_validation_non_pass_report.md
Please use this report to fix the remaining issues, then rerun the Validator Agent after the fixes. A few failures look like dependency/API mismatches in the Docker validation environment, some are harness issues, and some are weaker validation coverage warnings.
Feel free to discuss with me if anything in the report is unclear or if you think any item should be handled differently.
|
Hey @irvineoy thank you for the review and the changes. I am not able to access pr45_validation_non_pass_report.md, it shows 404 error. |
…eal references - Guard unimplemented FlyDSL kernel.py stubs (NotImplementedError -> SKIP) in model-only torch2flydsl correctness and benchmark paths; keep validating model.py against the aiter op at full tolerance. - Fix gemm_a8w8_bpreshuffle for flydsl 0.2.2 (cast idx2crd inputs to i32). - Add real numerical references to 11 triton2flydsl harnesses that were finiteness-only (mha, mla, unified_attention[+sparse_mla], fp8_mqa_logits, moe_fused_gemm, fav3_sage, moe_routing_sigmoid_top1, jagged_dense_bmm_broadcast_add, rmsnorm). - moe_topk_softmax: compare absolute matched-id weight error and document the intrinsic bf16 routing-floor exception for biased large-E shapes. - Resolve validator WARNs (per_tensor_fp8_quant, gelu_fast, rope_thd_fwd, aiter/rmsnorm large-N shape) and add the silu_and_mul benchmark stub-guard. Co-authored-by: Cursor <cursoragent@cursor.com>
b069e2b to
137b0ba
Compare
|
Ah there were 2 commits added by Vincent Ouyang after which I force pushed some changes. I think I may have overwritten those. |
Introduce the torch2flydsl task type (a PyTorch `class Model` in KernelBench format as the source, a FlyDSL kernel as the target) and register both the torch2flydsl and triton2flydsl task types in the prompt builder. Add the torch2flydsl task corpus: pure-torch, AMD-runtime-faithful references validated against the real aiter ops at a tight gate, covering GEMM, MoE routing/gating/sorting, normalization, RoPE, activation, and quantization. FP8 references select the arch-appropriate dtype (e4m3fnuz on gfx942, e4m3fn on gfx950); FP4/MX-only tasks are tagged gfx950-only with a harness skip-guard. Routing is made deterministic so the reference and the op select the same experts. Ignore build/ artifacts.
Standalone Triton-source tasks (depend only on triton/torch) faithfully extracted from aiter, covering bf16/fp8 GEMM (incl. block-scale, batched, feed-forward), MoE routing and grouped GEMM, normalization, softmax, RoPE, quantization, and attention (MHA / MLA / unified / sage / fp8 MQA logits). Each task ships the Triton source, a harness that runs the Triton kernel and gates it against an in-source reference at the upstream tolerance, and a config. MX scaled-dot tasks are tagged gfx950-only with a skip-guard. Upstream license headers are retained.
Standalone Triton-source tasks faithfully extracted from SGLang's linear-attention / Gated-DeltaNet chunk pipeline: chunk state update (chunk_fwd_h), chunk output (chunk_fwd_o), fused recurrent decode, and the L2-norm forward. Each ships the Triton source, a harness that runs the Triton kernel against an in-source reference at the upstream tolerance, and a config. Upstream license headers are retained.
Standalone Triton-source task faithfully extracted from SGLang's fused-MoE grouped-GEMM kernel, with a harness that runs the Triton kernel against an in-source reference at the upstream tolerance, and a config. Upstream license headers are retained.
Standalone Triton-source tasks faithfully extracted from SGLang: the native RadixAttention prefill/decode/extend backend and state-merge, the rest of the Gated-DeltaNet chunk pipeline (gating, cumsum, scaled-dot-kkt, norm-gate, wy_fast), Mamba2 SSD chunk-state, lightning linear attention, MoE routing, gated and dual-residual normalization, expert combine, multimodal RoPE, and FP4 indexer quantization. Each ships the Triton source, a harness that runs the Triton kernel against an in-source reference at the upstream tolerance, and a config. Upstream license headers are retained.
Standalone Triton-source tasks faithfully extracted from the Meta generative-recommenders (HSTU) project: jagged-dense BMM with broadcast add, jagged-dense broadcast add, layer norm, and fused SwiGLU. Each ships the Triton source, a harness that runs the Triton kernel against an in-source reference at the upstream tolerance, and a config. Upstream license headers are retained.
…eal references - Guard unimplemented FlyDSL kernel.py stubs (NotImplementedError -> SKIP) in model-only torch2flydsl correctness and benchmark paths; keep validating model.py against the aiter op at full tolerance. - Fix gemm_a8w8_bpreshuffle for flydsl 0.2.2 (cast idx2crd inputs to i32). - Add real numerical references to 11 triton2flydsl harnesses that were finiteness-only (mha, mla, unified_attention[+sparse_mla], fp8_mqa_logits, moe_fused_gemm, fav3_sage, moe_routing_sigmoid_top1, jagged_dense_bmm_broadcast_add, rmsnorm). - moe_topk_softmax: compare absolute matched-id weight error and document the intrinsic bf16 routing-floor exception for biased large-E shapes. - Resolve validator WARNs (per_tensor_fp8_quant, gelu_fast, rope_thd_fwd, aiter/rmsnorm large-N shape) and add the silu_and_mul benchmark stub-guard. Co-authored-by: Cursor <cursoragent@cursor.com>
137b0ba to
ce9371c
Compare
3 participants
Summary
Adds two task suites for FlyDSL kernel translation:
torch2flydsl(45 tasks): PyTorchModelreferences in KernelBench format for AMD-runtime operations, validated against independent AITER oracles.triton2flydsl(51 tasks): standalone Triton-source kernels whose harnesses execute the Triton target against an independent in-source reference.Together they cover GEMM (including FP8/block-scale/FP4 variants), attention/MLA, MoE routing, normalization, RoPE, quantization, and linear-attention/SSD. The PR also:
gfx950runtime tolmsysorg/sglang-rocm:v0.5.14-rocm720-mi35x-20260705(AITER 0.1.17 / FlyDSL 0.2.2);/tmp/aiter_configsonly for that exactgfx950runtime;torch2flydslstarter contract: task-package validation may accept an explicit pureNotImplementedErrorstarter after its independent oracle passes, while the optimization evaluator rejects missing or unimplemented declared targets before scoring;Deferred MoE split
The three heavyweight FlyDSL MoE ports were removed from this PR and preserved byte-for-byte in Draft PR #58:
torch2flydsl/moe_a8w4_kerneltorch2flydsl/moe_kerneltorch2flydsl/moe_swiglu_kernelThey have real FlyDSL lowering work remaining and are not part of this PR's merge gate.
Validation
Validated on MI355X /
gfx950, non-root, using the pinned v0.5.14 image with isolated writable caches:performance_report.jsonfiles were valid and contained finite positive latency (starter reports are explicitly reference fallback, never claimed as target latency);make check-docker-runnerpassed;make check-evaluatorpassed 9/9 tests, including pure/conditional starter classification and missing-target rejection.The previously passing 69-task subset was independently rerun read-only across six GPUs: 69/69 tasks, 207/207 configured stages, and 69/69 valid performance reports.
Sources and licenses
Kernels are extracted from upstream AITER, SGLang, and Meta generative-recommenders sources. Each file retains its upstream license/copyright header; task-specific changes make kernels standalone and provide correctness/performance harnesses for the arena.