Skip to content

[WebGPU] Unify core implementations of GEMM and MatMul #24586

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 20 commits into from
Jun 2, 2025

Conversation

xiaofeihan1
Copy link
Contributor

@xiaofeihan1 xiaofeihan1 commented Apr 29, 2025

Description

This PR extract core implementations into gemm_utils.cc which is used to generate shader both GEMM and Matmul ops. The core implemenations included scalar and vec4 versions of GEMM and Matmul.

Motivation and Context

There are many common codes for GEMM and Matmul, so we want to extra common code to unify their implementations.
Blank diagram (1)

@xiaofeihan1 xiaofeihan1 changed the title [WIP][WebGPU] Add vec1 implementation for GEMM [WIP][WebGPU] Unify core implementations of GEMM and MatMul May 12, 2025
@xiaofeihan1 xiaofeihan1 marked this pull request as ready for review May 14, 2025 02:18
@xiaofeihan1 xiaofeihan1 changed the title [WIP][WebGPU] Unify core implementations of GEMM and MatMul [WebGPU] Unify core implementations of GEMM and MatMul May 14, 2025
@fs-eire fs-eire added the ep:WebGPU ort-web webgpu provider label May 14, 2025
@xiaofeihan1 xiaofeihan1 requested a review from qjia7 May 16, 2025 09:17
qjia7
qjia7 previously approved these changes May 19, 2025
Copy link
Contributor

@qjia7 qjia7 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.
@fs-eire @guschmue Please take a look, thanks.

@qjia7 qjia7 requested review from fs-eire and guschmue May 19, 2025 01:40
@guschmue
Copy link
Contributor

testing it with a bunch of models that use Gemm

@guschmue
Copy link
Contributor

/azp run Linux QNN CI Pipeline,Win_TRT_Minimal_CUDA_Test_CI,Windows ARM64 QNN CI Pipeline,Windows GPU Doc Gen CI Pipeline,Windows x64 QNN CI Pipeline

Copy link

Azure Pipelines successfully started running 5 pipeline(s).

qjia7
qjia7 previously approved these changes May 27, 2025
guschmue
guschmue previously approved these changes May 27, 2025
@fs-eire
Copy link
Contributor

fs-eire commented May 27, 2025

The following test cases are failing on QNN EP:

2025-05-27T17:02:22.9260730Z 1: [  FAILED  ] GemmOpTest.GemmOptimizePacked
2025-05-27T17:02:22.9260956Z 1: [  FAILED  ] GemmOpTest.GemmOptimizePackedTransA
2025-05-27T17:02:22.9261190Z 1: [  FAILED  ] GemmOpTest.GemmOptimizePackedTransB
2025-05-27T17:02:22.9261428Z 1: [  FAILED  ] GemmOpTest.GemmOptimizePackedTransAB

@xiaofeihan1 xiaofeihan1 dismissed stale reviews from guschmue and qjia7 via a81be33 May 28, 2025 00:09
@fs-eire
Copy link
Contributor

fs-eire commented May 28, 2025

Some CI jobs didn’t run. Trying to trigger them by close and re-open the PR.

@fs-eire fs-eire closed this May 28, 2025
@fs-eire fs-eire reopened this May 28, 2025
@guschmue
Copy link
Contributor

/azp run Linux QNN CI Pipeline,Win_TRT_Minimal_CUDA_Test_CI,Windows ARM64 QNN CI Pipeline,Windows GPU Doc Gen CI Pipeline,Windows x64 QNN CI Pipeline

Copy link

Azure Pipelines successfully started running 5 pipeline(s).

@guschmue
Copy link
Contributor

/azp run Test Linux CUDA x64 Release,Test Linux TensorRT x64 Release,web_Debug / build_onnxruntime_web,web_Release / build_onnxruntime_web

Copy link

No pipelines are associated with this pull request.

@guschmue
Copy link
Contributor

/azp run web_Release / build_onnxruntime_web

Copy link

No pipelines are associated with this pull request.

@guschmue
Copy link
Contributor

I fear you need to merge with main - some definitions for the CI have changed :(

@xiaofeihan1
Copy link
Contributor Author

It seems that Lint pipeline is broken with the latest main.

@fs-eire
Copy link
Contributor

fs-eire commented May 29, 2025

There are some known issues and it's being fixed. please wait for the fixes merged into main.

@guschmue
Copy link
Contributor

/azp run web_Debug / build_onnxruntime_web

Copy link

No pipelines are associated with this pull request.

@fs-eire
Copy link
Contributor

fs-eire commented May 30, 2025

Merged latest main branch, which contains the CI pipeline fixes.

@fs-eire
Copy link
Contributor

fs-eire commented May 30, 2025

/azp run Linux QNN CI Pipeline,Win_TRT_Minimal_CUDA_Test_CI,Windows ARM64 QNN CI Pipeline,Windows GPU Doc Gen CI Pipeline,Windows x64 QNN CI Pipeline

Copy link

Azure Pipelines successfully started running 5 pipeline(s).

@fs-eire fs-eire requested review from guschmue and qjia7 May 30, 2025 13:55
@guschmue guschmue merged commit 915a999 into microsoft:main Jun 2, 2025
86 of 87 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ep:WebGPU ort-web webgpu provider
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants