Skip to content

Split compiletests and difftests into parallel CI jobs #272

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Jun 5, 2025

Conversation

LegNeato
Copy link
Collaborator

@LegNeato LegNeato commented Jun 4, 2025

We could have them kick off after the main job and reuse artifacts, but it seems better to just kick them all off at once?

@LegNeato
Copy link
Collaborator Author

LegNeato commented Jun 4, 2025

End-to-end CI Times:

  • Before difftests: ~27 mins
  • After difftests: ~38 mins
  • This PR: ~21 mins

@LegNeato LegNeato enabled auto-merge June 4, 2025 21:47
@Firestar99
Copy link
Member

I'd only make them depend on each other if we run out of ci runtime tokens, but they're free for open source projects so... who cares.

I've added two more things:

  • separated the android build out into it's own thing
  • check the lockfile in difftests for uncommitted changes

@Firestar99
Copy link
Member

Firestar99 commented Jun 5, 2025

baseline: ~22 min

Most of our CI is now below 10min, apart from the main test. The slowest component is the workspace test with 7m on ubuntu/mac and 10m on windows. I think most of that is coming from release compiling our examples, which depend on ash, wgpu and winit, so I excluded them from testing. They don't have any tests anyway, so the following check on examples should be sufficient.
-> ~23 min regression?, reverting for now

Also it doesn't have "use-installed-tools"!
-> ~19 min, linux & windows: -3min, mac: -7min

We also need to exclude our new difftests, as they too depend on wgpu. I moved their testing to our difftest ci and readded the examples exclusion.
-> ~16 min min

windows test is the slowest with 16min, followed closely by windows difftest with 15min. Comparing to ubuntu and mac:

  • cargo fetch takes 1min, whereas it takes <7s on unix
  • build example shaders (in release) seems to be a clean build on windows with 2m30, where it's an incremental on unix with ~30s
  • running the difftest does not reuse artifacts from difftest/lib due to being run in a different workspace
  • in general windows is 1.5-2x slower in building rust, but that's expected

@Firestar99 Firestar99 disabled auto-merge June 5, 2025 10:11
Copy link
Member

@Firestar99 Firestar99 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm giving you the 👍 but disabled auto-merging so you can have another look at my changes

@LegNeato
Copy link
Collaborator Author

LegNeato commented Jun 5, 2025

I'm fine with the changes, though I think it is cleaner to split by "type" (unit, diff, compile) rather than by crate (e.g. the difftest job running difftest and their associated unit tests).

But this isn't too complicated yet so it doesn't really matter either way.

@LegNeato LegNeato added this pull request to the merge queue Jun 5, 2025
Merged via the queue into Rust-GPU:main with commit 77070b7 Jun 5, 2025
13 checks passed
@LegNeato LegNeato deleted the testci branch June 5, 2025 10:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants