Skip to content

forward performance tuning for MI350 #4925

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 28 commits into
base: main
Choose a base branch
from
Open

forward performance tuning for MI350 #4925

wants to merge 28 commits into from

Conversation

liligwu
Copy link
Contributor

@liligwu liligwu commented Sep 24, 2025

Fix numerical issues
forward performance tuning for MI350

avbokovoy and others added 27 commits September 24, 2025 00:49
@avbokovoy @liligwu
Add gfx950 build support + fp16 fix + index type fix
fe4d8f0
@avbokovoy @liligwu
Change int64_t to index_t as template parameters in load_raw_per_warp
2006f08
@avbokovoy @liligwu
Implement llvm fp16 buffer load for gfx950
757a2f4
@avbokovoy @liligwu
Fix c-style half to float cast
f875d54
@avbokovoy @liligwu
Patch 256 half stores
ea9d8f8
@shbiswas834 @liligwu
cta_per_row workgroup optim
e63ead2
@shbiswas834 @liligwu
Added mi350 guards
b9eebb4
@shbiswas834 @liligwu
Fix index overflow in row load
69ae10e
@shbiswas834 @liligwu
cta_per_row workgroup reduce by 4 optim
8f692dc
@avbokovoy @liligwu
Fix mixed_D frontend to backend connection
768dc01
@kudomcho @liligwu
changed max_segment_length_per_cta to 4096
151d2dd
@shbiswas834 @liligwu
added rocm guards and removed comment
54e0e24
@liligwu
clean debug statements in Hip.cmake
7b2684c
@shbiswas834 @liligwu
Merge pull request #121
9b22e17 
warp per row wg change
@avbokovoy @liligwu
Guard f16 llvm intrinsics with ROCm >=7.0
9adc6bc
@liligwu
fix the bug in dimention 160 in ROCm optimization
dc16185
@avbokovoy @liligwu
Cleanup optimized warp_per_raw kernel
00c1914
@avbokovoy @liligwu
Add 320 embedding dim support for optimized warp_per_row kernel
f2c662a
@liligwu
changed the max length per warp and cta per row WG size
f8fe9d7
@kudomcho @liligwu
added DPP and changed max length per warp to 16k
279aeac
@kudomcho @liligwu
guard max segment warp based on emb dim
d59e3d6
@kudomcho @liligwu
added guarding opt of max segment for the case batch size list=1
faf378e
@liligwu
opt for grad_indice_weights kernel
d9239e9
@kudomcho @liligwu
added store row per warp on emb 192 and added accuracy test functiona…
145a673 
…lity
@shbiswas834 @liligwu
workgroup tuning and loop unrolled
986cceb
@Hardcode84 @liligwu
specialize
2bf70c6
@liligwu
explicitly link to tbb
4d2bfdd
@meta-cla meta-cla bot added the cla signed label Sep 24, 2025
@netlify Netlify
Copy link

netlify bot commented Sep 24, 2025 •
edited
Loading

✅ Deploy Preview for pytorch-fbgemm-docs ready!

Name Link
🔨 Latest commit f10335a
🔍 Latest deploy log https://app.netlify.com/projects/pytorch-fbgemm-docs/deploys/68d59288375f99000859393d
😎 Deploy Preview https://deploy-preview-4925--pytorch-fbgemm-docs.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@facebook-github-bot
Copy link
Contributor

@haoyuz has imported this pull request. If you are a Meta employee, you can view this in D83116315.

@q10
Copy link
Contributor

q10 commented Oct 2, 2025

@liligwu we're seeing

OSError: libtbb.so.12: cannot open shared object file: No such file or directory

We already install tbb here, so it might just be an issue of updating the build scripts to put libtbb in the LD_LIBRARY_PATH

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
cla signed module: rocm
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@liligwu @facebook-github-bot @q10 @avbokovoy @shbiswas834 @kudomcho @Hardcode84