Skip to content

Skip unused CI suites before setup and scope tool installation - #2185

Open
dblock wants to merge 1 commit into
trailofbits:masterfrom
dblock:dblock/ci-setup
Open

dblock wants to merge 1 commit into
trailofbits:masterfrom
dblock:dblock/ci-setup

Conversation

@dblock

@dblock dblock commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Select CI suites before allocating test runners. Ordinary PRs retain all ten regular matrix entries without starting the twelve expensive jobs that previously installed tools, cleaned disks, and fetched dependencies before exiting without running tests. Changes under expensive/, scheduled runs, and manual runs retain the full expensive matrix. Merge-queue selection uses the merge group's base SHA.

Restrict setup to the suites that need it: install cargo-hack, cargo-udeps, Prettier, and the cargo-rdme compatibility toolchain only for the CI suite; install nightly rustfmt for CI and example-formatting tests; install supply-chain tools only when scheduled checks will run. Replace the broad Rustup update with a stable-only update and leave nightly installation to the existing toolchain-preinstallation helper.

Keep the actual test commands, all three host platforms, dependency caches, disk cleanup, and preinstallation needed to avoid concurrent Rustup operations. Include planner failures in the final required-check gate.

Validation

Passed locally: actionlint, the existing ShellCheck test, formatting, workspace custom lints with all features/tests, and planner regressions covering PRs, merge groups, scheduled/manual runs, deleted/renamed paths, and invalid or unavailable base commits.

Timing Comparison

Both runs completed successfully: before, unchanged workflow, and after, this PR at 8ad76c5.

Metric Before After
Test matrix jobs 22 10
Elapsed time in testless expensive jobs, summed 83m 02s 0
All job elapsed times, summed 7h 24m 56s 5h 37m 46s
Setup steps across the ten retained jobs, summed 75m 29s 72m 25s
Ubuntu CI-suite setup 10m 29s 7m 53s
Workflow elapsed time 75m 31s 70m 37s

Observed reductions: 107m 10s (24.1%) in summed job elapsed time, 3m 04s (4.1%) in retained-job setup, and 4m 54s (6.5%) in workflow elapsed time. The new planner took seven seconds. The twelve removed jobs previously spent only six seconds combined in their final Test steps; the rest was setup and job overhead.

Summed job times are elapsed durations, not OS-weighted billing minutes. Setup is the sum of completed step durations preceding Test; workflow elapsed time runs from workflow creation through the last job's completion. The overall job and workflow totals include the new planner.

This is one observational pair, not a controlled benchmark: the baseline is #2179, which also adds four integration tests absent from this independent, master-based PR, and runner/cache conditions vary. Do not attribute all test-runtime differences to these workflow changes. The structural saving is avoiding twelve testless jobs. Long-running tests remain the critical path: the after-run Windows CLI job took 69m 46s, including 59m 51s in its unchanged Test step.

Hosted CI passed all ten selected platform/suite jobs, the planner, and the final gate.

AI-authored on behalf of @dblock.

Plan explicit matrix entries from the PR or merge-group base commit before allocating test runners. Preserve regular platform coverage and full expensive suites on relevant changes, schedules, and manual runs. Fail the required gate if planning fails.

Install formatting, CI, and scheduled-check tools only for suites that use them. Consolidate Rustup updates while retaining common toolchain preinstallation and disk cleanup. Test event selection, deleted and renamed paths, and invalid bases.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant