feat(09): in-kernel SM profiler + CuTe DSL line-info & ncu profiling - #22
Merged
Conversation
Reusable, kernel-agnostic in-kernel profiler (cutedsl_sm_profiler.py) plus a profiled pipelining GEMM (cutedsl_pipelining_profiled.py): %globaltimer-stamped phase/per-tile events exported as a Perfetto trace, gated by a compile-time flag so the production kernel is unchanged when profiling is off. Hot-path overhead cut ~50% (721 -> 360 ns/k-tile on B200) by per-warp counter cache-line padding (eliminates cross-warp false sharing); type-keyed event pairing via a compile-time open-id map (zero runtime cost) and a fused event_switch. Verified RE 0.01% with stable Perfetto slice count. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…dsl in profile.sh Add --generate-line-info to every cutedsl_*.py cute.compile so Nsight Compute can map SASS back to the Python source (pair with ncu --import-source yes). Each profile.sh now comments out the C++/reference ncu command and adds a CuTe DSL one targeting cutedsl_<stem>.py via --kernel-name regex:.*<stem>.*. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Two pieces of work on the
09CuTe DSL track.1. In-kernel SM timeline profiler (
e00197a)cutedsl_sm_profiler.py— reusable, kernel-agnostic in-kernel profiler: per-(block,warp) append buffer,%globaltimerrange events, Perfetto export. Callers register their own event types and close events by type (event_end(PROF_X)) via a compile-time open-id map (zero runtime cost); a fusedevent_switchshares one timer read at adjacent boundaries.cutedsl_pipelining_profiled.py— the pipelining GEMM instrumented with it, compile-time gated so the production kernel is unchanged when profiling is off (kept diff-aligned withcutedsl_pipelining.py)..cs) and smem-counter variants were ablated on G5 and found neutral-or-worse (376 / 414 ns/k-tile), so they were removed — only the optimal implementation remains. Verified RE 0.01% with a stable Perfetto slice count throughout.2. line-info + ncu profiling across all CuTe DSL impls (
74ef8e1)--generate-line-infoto everycutedsl_*.pycute.compile, so Nsight Compute can map SASS back to the Python source (use withncu --import-source yes).profile.shnow profiles the CuTe DSL implementation: the old C++/referencenculine is commented out and a CuTe DSL one is added, e.g.ncu -o ncu_prof_9 --import-source 1 --set full --kernel-name "regex:.*pipelining.*" -f python cutedsl_pipelining.py(10-gemm-api has no CuTe DSL impl, left as-is).Verification
All
cutedsl_*.pypy_compileclean; the profiler path validated on G5/B200 (RE 0.01%, 1081344 Perfetto slices, 360 ns/k-tile).🤖 Generated with Claude Code