Update all non-deterministic tests to run 3x the normal CI/CD workflow on the release validation pipeline #3458
Conversation
matthewige
requested review from
dthaler,
poornagmsft,
Alan-Jowett,
saxena-anurag,
shankarseal,
dv-msft,
gtrevi,
shpalani,
mtfriesen and
rectified95
as code owners
|
All, is there any way I can test this without merging this PR? (I could not seem to schedule the release validation CI on this branch). |
To schedule, you need to have "signed-off" added under Labels in the Issue. (Ref: #3261). To test you can 1. temp. create an issue and add the label by yourself, 2. use the existing issue to add the label temp. to trigger. |
matthewige
added
signed-off
| needs: regular | ||
| if: github.event_name == 'workflow_dispatch' | ||
| uses: ./.github/workflows/reusable-test.yml | ||
| with: | ||
| name: km_mt_stress_tests | ||
| name: km_mt_stress_tests-1 | ||
| pre_test: .\setup_ebpf_cicd_tests.ps1 -KmTracing $true -KmTraceType "memory" |
There was a problem hiding this comment.
Why does this require 3 iterations here rather than making .\execute_ebpf_cicd_tests.ps1 have a parameter to handle it in there?
docs/ReleaseProcess.md
Outdated
| Once potential fixes are merged to the release branch repeat the process until the workflow completes successfully. | ||
|
|
||
| Note: If any non-deterministic tests (such as fuzzer or stress) are added in the future, the release validation workflow should be updated to run 3x the normal | ||
| CI/CD workflow (either 3x the duration or 3 instances of the test). |
There was a problem hiding this comment.
Is there any valid reason to run 3 instances of the test (as opposed to 3x the duration)? If so, what is the reason?
This reverts commit 4b20828.
|
@matthewige what is the status of this PR? Is it abandoned? If so, please close it. |
Description
Release managers were instructed to run the release validation workflow 3 times to ensure non-deterministic tests had enough time/attempts to run. This change updates the release validation to take care of this, and the manager should now only need to run the workflow once.
Closes #3376
Testing
None.
Documentation
Updated the release process documentation to indicate that only 1 run is required now.
Installation
n/a