ci(nightly): retire the dead macOS Colima job#422
Merged
Conversation
The job carried a "slow and flaky (infra)" charter, but the measured
record disagrees: its last 19 runs all failed deterministically in
17-40 seconds at `colima start`, and the scan step never executed. On
the hosted arm64 runner neither backend boots — vz exits immediately
(no nested virtualization on hosted Apple Silicon), and a one-off
experiment (run 29487846488) showed lima 2.1.3's qemu driver panics
("send on closed channel") before the VM starts.
A permanently red best-effort job burns runner time and reads as
"occasionally passing" when it never does; its step summary promised a
scan result that was never produced. Remove it and record the evidence
and the re-add condition (hosted arm64 nested virtualization) in
place. macOS real-scan coverage remains a maintainer-run local check
on real Apple Silicon hardware, where Colima works.
haksungjang
added a commit
that referenced
this pull request
Jul 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
nightly's "macOS real scan (Colima)" was chartered as best-effort because Colima startup was believed "slow and flaky (infra)". Measurement says otherwise:
colima start --vm-type=vz— the scan step never executed once.Exiting:true, empty error array) — hosted Apple Silicon runners do not provide nested virtualization.send on closed channel, qemu_driver.go:412) before the VM starts.So the job cannot succeed on
macos-latestwith either backend, and itscontinue-on-error+ exclusion from the failure alarm made it invisible: a permanently red job burning a runner slot nightly while its comment claimed the result was "visible in the step summary" (the summary only ever printed the failure branch).What
report-failure.The experiment workflow lives only on a throwaway branch and is deleted with it — nothing lands in main.