Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speed up Axlearn CI #688

Merged
merged 9 commits into from
Oct 17, 2024
Merged

Conversation

soundway
Copy link
Contributor

@soundway soundway commented Sep 6, 2024

Make a few minor tweaks to make Axlearn CI faster:

  • Use async mode to run different parts of the test
  • Use worksteal to more evenly distribute the workload

This should cut CI time by roughly half.

Copy link
Contributor

@ruomingp ruomingp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Will defer to @markblee to approve.

@madrob
Copy link
Contributor

madrob commented Sep 11, 2024

We're using cimg/base:stable which is two years old and a deprecated tag. Looking at cimg/base:current which is more recent, that still only includes bash 5.1.16 - is that new enough for this feature?

@soundway
Copy link
Contributor Author

Yes, it seems like it. I just realized the wait feature I'm relying on requires Bash 5.1, not 5.2. So it's all good. I just updated the comment.

This change shouldn't affect this repo much other than having shorter test time due to having worksteal per each of the test shard. Given that pytype check in this repo seems to take the longest time, I recommend that we make build-and-test-job and pre-commit run in parallel, and maybe add more use machines with more CPU resources for the pre-commit check to speed things up.

@soundway
Copy link
Contributor Author

Ahh never mind, looks like I can just make those changes in .circleci/config.yml.

Make a few minor tweaks to make Axlearn CI faster:
  * Use async mode to run different parts of the test
  * Use worksteal to more evenly distribute the workload

This should cut CI time by roughly half.
@soundway soundway added this pull request to the merge queue Oct 17, 2024
Merged via the queue into apple:main with commit 27cc0f4 Oct 17, 2024
4 checks passed
@soundway soundway deleted the willsong/speedup_axlearn_ci branch October 17, 2024 19:30
kelvin-zou added a commit that referenced this pull request Oct 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants