Skip to content

feat(ci): Add Multi-Arch Release ASAN workflow#64

Merged
geomin12 merged 1 commit into
mainfrom
users/geomin12/release-asan
Jun 23, 2026
Merged

feat(ci): Add Multi-Arch Release ASAN workflow#64
geomin12 merged 1 commit into
mainfrom
users/geomin12/release-asan

Conversation

@geomin12

@geomin12 geomin12 commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

Adds a caller workflow for the ASAN release workflow in TheRock. Supports workflow_dispatch and scheduled nightly builds for Linux only.

Landed this ROCm/TheRock#5968 and propogating here. Progress on ROCm/TheRock#5937

ISSUE ID ROCm/TheRock#5937

Running a test here https://github.com/ROCm/TheRock/actions/runs/28049441058, but can get some reviews here at least

Adds a caller workflow for the ASAN release workflow in TheRock.
Supports workflow_dispatch and scheduled nightly builds for Linux only.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@geomin12 geomin12 requested a review from nunnikri June 23, 2026 05:07
@geomin12 geomin12 marked this pull request as ready for review June 23, 2026 21:45

# Trigger on a schedule to build nightly releases.
schedule:
# Runs at 12:00 AM UTC, which is 5:00 PM PDT (UTC-7)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shall we start later like 10PM or 12AM PDT, so that the normal nightly build and this doesnt conflict

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i'll keep this time in order to be consistent with

# Runs at 12:00 AM UTC, which is 5:00 PM PDT (UTC-7)

@geomin12 geomin12 merged commit 6767995 into main Jun 23, 2026
1 check passed
@geomin12 geomin12 deleted the users/geomin12/release-asan branch June 23, 2026 23:15
@ScottTodd

ScottTodd commented Jun 29, 2026

Copy link
Copy Markdown
Member

How was this tested prior to merge?

Running a test here https://github.com/ROCm/TheRock/actions/runs/28049441058, but can get some reviews here at least

This run failed.

I approved ROCm/TheRock#5968 conditionally on releases being sufficiently tested in dev before moving to nightly:

Need to test this with dev releases before we go straight to publishing weekly. This is very high risk.

The higher risk parts of this (jumping straight to nightly releases) were resolved already.

What we need to be careful about here is what was flagged on that PR: packages from ASan and non-ASan must not be mixed. The recent https://github.com/ROCm/rockrel/actions/runs/28341408029 workflow run did a few things wrong:

  • It built python packages then never copied them to the release bucket (python isn't tested/working yet, so that's just wasting some runner time)
  • It triggered pytorch and jax release jobs, which then used those python packages and uploaded asan jax packages to the release bucket: https://github.com/ROCm/rockrel/actions/runs/28355352077/job/83996952989#step:15:30, which overlapped with existing non-asan packages (overwriting them?) and then failed tests

@ScottTodd

Copy link
Copy Markdown
Member

Sent a revert #69

ScottTodd added a commit that referenced this pull request Jun 29, 2026
Reverts #64

This was not sufficiently tested in dev prior to enabling nightly builds
and it is overwriting at least non-asan JAX packages, see
#64 (comment)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants