feat(ci): support exhaustive tests for PRs (code changes or GH comment) #18327
+148
−25
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
🤖 GitHub commentsExpand to view the GitHub comments
Just comment with:
|
|
This pull request does not have a backport label. Could you fix it @v1v? 🙏
|
1 task
💛 Build succeeded, but was flaky
Failed CI Stepscc @v1v |
| - ELASTIC_SLACK_NOTIFICATIONS_ENABLE=false | ||
|
|
||
| - label: "Trigger logstash-exhaustive-tests-pipeline for GitHub comments" | ||
| if: build.env("GITHUB_PR_TRIGGER_COMMENT") != "" |
There was a problem hiding this comment.
What happens if a file has changed in the qa/acceptance/ and a comment in the PR is made?
will be the pipeline logstash-exhaustive-tests-pipeline executed twice?
There was a problem hiding this comment.
Two different events:
- push event with the file changeset
- github comment matches the values in https://github.com/elastic/logstash/pull/18327/files#diff-646855f9c36bcc3f7a5c49c1fa33be9b7771d99e5e2219332b215eaac6e51537R20-R21
Each event should be treated independently.
Otherwise, what do you mean by your question?
There was a problem hiding this comment.
Understood. We want to execute the tests in both cases and every time when push and the right files have changed or when a comment that matches has made.
Thanks!
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Release notes
rn:skip
What does this PR do?
This pull request introduces a new "smart exhaustive tests" pipeline for Logstash, integrating it into the Buildkite CI configuration and documenting it in the service catalog. The main goal is to trigger exhaustive test runs automatically for pull requests that modify files in the
qa/acceptance/directory or when specific GitHub comments are made, improving test coverage and automation for critical changes.Why is it important/What is the impact to the user?
Test the code that will rarely impact the standard PR test suite, but will exercise the acceptance tests.
Checklist
[ ] I have commented my code, particularly in hard-to-understand areas[ ] I have made corresponding changes to the documentation[ ] I have made corresponding change to the default configuration files (and/or docker env variables)[ ] I have added tests that prove my fix is effective or that my feature worksHow to test this PR locally
Add a path to
always_require_ci_on_changedin pull-request.json forlogstash-smart-exhaustive-tests-pipeline, then apply changes in that directory, commit and push. That will trigger the exhaustive tests pipeline in Buildkite.Caveats
The existing BK PR Bot does not support for code changes and GitHub comments, without running the first buidl when a PR is created.
That's the reason for creating the
logstash-smart-exhaustive-tests-pipelineto actually filter:This should help with running faster builds and reduce the overhead of running the exhaustive tests for the first build of each new PR.
Duplicates #18202, that I messed up with the rebase.