Skip to content

CMP-4326: Add CI jobs for periodic testing of the e2e tests in CaC-compliance operator repo - #80274

Open
Anna-Koudelkova wants to merge 1 commit into
openshift:mainfrom
Anna-Koudelkova:periodic_CaC_co
Open

CMP-4326: Add CI jobs for periodic testing of the e2e tests in CaC-compliance operator repo#80274
Anna-Koudelkova wants to merge 1 commit into
openshift:mainfrom
Anna-Koudelkova:periodic_CaC_co

Conversation

@Anna-Koudelkova

@Anna-Koudelkova Anna-Koudelkova commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

This should be an equivalent of current downstream periodic testing of CO.

Summary by CodeRabbit

This PR adds periodic CI Operator configurations to run AWS IPI end-to-end (e2e) tests for the ComplianceAsCode/compliance-operator repository across multiple OCP versions (4.12–4.22).

What changed, practically

  • Added new ci-operator config YAMLs under ci-operator/config/ComplianceAsCode/compliance-operator/ for ocp-4.12 through ocp-4.22 (one file per OCP variant).
  • For each OCP variant, the config defines a set of cron-scheduled AWS IPI e2e jobs using:
    • cluster_profile: aws-stackrox
    • workflow: ipi-aws
    • cli: latest
    • shared environment inputs (e.g., BASE_DOMAIN, CATALOG_SOURCE_NAME, OO_BUNDLE, OO_INSTALL_NAMESPACE, TEST_TYPE=release, and catalog/bundle refs)
    • the same two test catalog references
  • Each job runs a different suite-specific command from src:
    • make e2e-prerelease
    • make e2e-parsing
    • make e2e-deployment
    • make e2e-scan-config
    • make e2e-tailoring
    • make e2e-cel
  • Per-job resource handling:
    • global defaults set resources['*'] to cpu: 200m and memory: 400Mi
    • the test step overrides to cpu: 100m
  • Cron timing differs per suite and OCP version (with suite-specific offsets visible in job naming/scheduling such as -f14, -f28, -f60 across variants).
  • Each config ends with zz_generated_metadata (branch master, org ComplianceAsCode, repo compliance-operator, variant ocp-<version>).

Infrastructure impact

  • Extends periodic CI coverage for compliance-operator by adding recurring e2e validation runs on OCP 4.12–4.22 using a consistent AWS IPI + Stackrox-based execution profile.
  • Improves regression/integration detection across both operator behavior and e2e compliance workflows by splitting suites into separately scheduled jobs.

PR metadata & review notes

  • Marked as work-in-progress; WIP label present.
  • Jira validity for CMP-4326 was confirmed, with a note that the target version was not set as expected for the intended branch.
  • Rehearsal command was issued during review for the OCP 4.22 e2e job.

@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Jun 9, 2026
@openshift-ci-robot

openshift-ci-robot commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

@Anna-Koudelkova: This pull request references CMP-4326 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "5.0.0" version, but no target version was set.

Details

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci openshift-ci Bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 9, 2026
@openshift-ci
openshift-ci Bot requested review from Vincent056 and taimurhafeez June 9, 2026 08:21
@openshift-ci

openshift-ci Bot commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Anna-Koudelkova

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 9, 2026
@coderabbitai

coderabbitai Bot commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Walkthrough

Adds CI Operator YAML configurations for ComplianceAsCode across OCP 4.12–4.22, covering release selection, resource defaults, scheduled AWS IPI E2E suites, and generated variant metadata.

Changes

ComplianceAsCode CI configurations

Layer / File(s) Summary
Release configuration and resource defaults
ci-operator/config/ComplianceAsCode/compliance-operator/...__ocp-4.12.yaml through ...__ocp-4.22.yaml
Each variant defines a build_root, OCP release targets, and default CPU and memory requests.
Scheduled AWS E2E workflows
ci-operator/config/ComplianceAsCode/compliance-operator/...__ocp-4.12.yaml through ...__ocp-4.22.yaml
Each configuration adds scheduled AWS ipi-aws jobs sharing cluster, environment, catalog, and resource settings while selecting suite-specific make e2e-* commands.
Generated variant metadata
ci-operator/config/ComplianceAsCode/compliance-operator/...__ocp-4.12.yaml through ...__ocp-4.22.yaml
Each file records the master branch, organization, repository, and OCP variant metadata.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Suggested reviewers: taimurhafeez, vincent056

🚥 Pre-merge checks | ✅ 15
✅ Passed checks (15 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed PR only adds ci-operator YAML configs; no Ginkgo test titles were added or modified.
Test Structure And Quality ✅ Passed PR only adds ci-operator YAML configs for periodic jobs; no Ginkgo/Go test code or test blocks were changed, so this check is not applicable.
Microshift Test Compatibility ✅ Passed PR only adds ci-operator YAML jobs; no Ginkgo test code or MicroShift-specific test bodies were added to inspect.
Single Node Openshift (Sno) Test Compatibility ✅ Passed PR only adds ci-operator YAML jobs; no new Ginkgo test code or SNO-sensitive assertions were introduced.
Topology-Aware Scheduling Compatibility ✅ Passed Only ci-operator periodic-test YAMLs were added; searches found no topology-sensitive scheduling fields, node selectors, affinity, anti-affinity, PDBs, or spread constraints.
Ote Binary Stdout Contract ✅ Passed Only ci-operator YAML config files changed; no process-level Go/binary code was modified, so the stdout contract check is not applicable.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed Only ci-operator YAML jobs were added; no Ginkgo/e2e test code changed, and no IPv4/public-internet assumptions are present in the diff.
No-Weak-Crypto ✅ Passed Scanned all 11 added YAMLs; they only define CI jobs and contain no MD5/SHA1/DES/RC4/3DES/Blowfish/ECB or custom crypto.
Container-Privileges ✅ Passed Changed YAMLs only define ci-operator/periodic jobs; scans found no privileged:true, hostPID/Network/IPC, SYS_ADMIN, allowPrivilegeEscalation, or root-only settings.
No-Sensitive-Data-In-Logs ✅ Passed The PR only adds CI job YAMLs; I found no logging statements or secret/token/password/PII output paths in the changed files.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the addition of periodic CI jobs for e2e testing in the compliance-operator repo.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (3)
ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.12.yaml (2)

35-35: ⚡ Quick win

Verify bundle image strategy: all OCP versions test master branch bundle.

All test jobs across all OCP versions (4.12–4.22) reference the same bundle image with the :master tag. This means version-specific periodic tests will always deploy and test the latest master-branch operator bundle, regardless of which OCP version the test targets.

If the intent is to test the compliance-operator's master branch compatibility across multiple OCP versions, this is correct. However, if version-specific tests should use version-specific operator bundles (e.g., a 4.12-compatible operator for OCP 4.12 tests), the bundle references should be updated accordingly.

Also applies to: 56-56, 77-77, 98-98, 119-119, 140-140, 161-161

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.12.yaml`
at line 35, The OO_BUNDLE variable is hardcoded to the master-tagged bundle
(OO_BUNDLE:
quay.io/redhat-user-workloads/ocp-isc-tenant/compliance-operator-bundle-dev:master),
causing all OCP-version jobs to deploy the same master bundle; update the job
definition to select a version-appropriate bundle instead (e.g., replace the
literal :master with a variable or construct that uses the job's OCP version
like :4.12 or a derived ${OCP_VERSION}-compatible tag), and apply the same
change to the other occurrences (lines referencing OO_BUNDLE at the noted
locations) so periodic tests for each OCP version pull the correct operator
bundle for that target version.

29-29: ⚡ Quick win

Consider staggering cron schedules to avoid cluster overload.

All 56 periodic jobs across the 8 configuration files (7 tests × 8 OCP versions) share the identical cron schedule 12 15 1,15 * *, causing them to execute simultaneously on the 1st and 15th of each month at 15:12 UTC. Launching 56 AWS IPI clusters concurrently may overwhelm the CI infrastructure or exhaust the quay-aws cluster profile quota.

Consider staggering the schedules across the day or across files (e.g., 4.12 at 12:15, 4.13 at 13:15, etc.) to distribute load.

Also applies to: 50-50, 71-71, 92-92, 113-113, 134-134, 155-155

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.12.yaml`
at line 29, Multiple periodic jobs use the identical cron entry "cron: 12 15
1,15 * *" causing 56 jobs to run concurrently; update the cron lines in the
affected config entries (look for "cron: 12 15 1,15 * *" in
ComplianceAsCode-compliance-operator-master__ocp-4.12.yaml and the other files
referenced) to stagger execution times — e.g., increment the hour per OCP
version (4.12 -> "12 15", 4.13 -> "13 15", etc.) or distribute across different
minutes/hours so jobs no longer all run at 15:12 UTC, ensuring each file’s cron
expression is unique and evenly spreads load.
ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.19.yaml (1)

28-174: ⚖️ Poor tradeoff

Consider consolidating duplicate test definitions across OCP versions.

All seven E2E test jobs (prerelease, parallel, serial, deployment, config, tailoring, CEL) are identically configured across all OCP variant files, differing only in the target OCP version. This pattern is typical for variant periodic configs supporting CI analytical tooling, but consider whether step-registry workflows or shared test definitions could reduce maintenance overhead as the test suite evolves.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.19.yaml`
around lines 28 - 174, The CI file repeats seven nearly identical jobs
(e2e-aws-prerelease-f14, e2e-aws-parallel-f14, e2e-aws-serial-f14,
e2e-aws-deployment-f14, e2e-aws-config-f14, e2e-aws-tailoring-f14,
e2e-aws-cel-f14); factor out the common steps/env/test definitions into a shared
template or step-registry entry and reference it from each variant (or use YAML
anchors/aliases) so only the OCP-specific differences remain in the variant
files; update the job entries (the as: job names and their steps/test blocks) to
import/extend the common definition instead of duplicating the full blocks.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In
`@ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.15.yaml`:
- Around line 1-5: The build root tag is pointing to the wrong OCP version:
update build_root.image_stream_tag.tag (the tag value currently contains
"openshift-4.19") to the matching OCP 4.15 release tag (replace "openshift-4.19"
with "openshift-4.15" in the tag string) so the build uses the correct base
environment for the ocp-4.15 variant.

---

Nitpick comments:
In
`@ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.12.yaml`:
- Line 35: The OO_BUNDLE variable is hardcoded to the master-tagged bundle
(OO_BUNDLE:
quay.io/redhat-user-workloads/ocp-isc-tenant/compliance-operator-bundle-dev:master),
causing all OCP-version jobs to deploy the same master bundle; update the job
definition to select a version-appropriate bundle instead (e.g., replace the
literal :master with a variable or construct that uses the job's OCP version
like :4.12 or a derived ${OCP_VERSION}-compatible tag), and apply the same
change to the other occurrences (lines referencing OO_BUNDLE at the noted
locations) so periodic tests for each OCP version pull the correct operator
bundle for that target version.
- Line 29: Multiple periodic jobs use the identical cron entry "cron: 12 15 1,15
* *" causing 56 jobs to run concurrently; update the cron lines in the affected
config entries (look for "cron: 12 15 1,15 * *" in
ComplianceAsCode-compliance-operator-master__ocp-4.12.yaml and the other files
referenced) to stagger execution times — e.g., increment the hour per OCP
version (4.12 -> "12 15", 4.13 -> "13 15", etc.) or distribute across different
minutes/hours so jobs no longer all run at 15:12 UTC, ensuring each file’s cron
expression is unique and evenly spreads load.

In
`@ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.19.yaml`:
- Around line 28-174: The CI file repeats seven nearly identical jobs
(e2e-aws-prerelease-f14, e2e-aws-parallel-f14, e2e-aws-serial-f14,
e2e-aws-deployment-f14, e2e-aws-config-f14, e2e-aws-tailoring-f14,
e2e-aws-cel-f14); factor out the common steps/env/test definitions into a shared
template or step-registry entry and reference it from each variant (or use YAML
anchors/aliases) so only the OCP-specific differences remain in the variant
files; update the job entries (the as: job names and their steps/test blocks) to
import/extend the common definition instead of duplicating the full blocks.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 1e5128a7-fdb2-41dd-bf16-3bbd02ad4162

📥 Commits

Reviewing files that changed from the base of the PR and between df0ba54 and cb67ab4.

⛔ Files ignored due to path filters (1)
  • ci-operator/jobs/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master-periodics.yaml is excluded by !ci-operator/jobs/**
📒 Files selected for processing (11)
  • ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.12.yaml
  • ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.13.yaml
  • ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.14.yaml
  • ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.15.yaml
  • ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.16.yaml
  • ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.17.yaml
  • ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.18.yaml
  • ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.19.yaml
  • ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.20.yaml
  • ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.21.yaml
  • ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.22.yaml

Comment on lines +1 to +5
build_root:
image_stream_tag:
name: release
namespace: openshift
tag: rhel-9-release-golang-1.23-openshift-4.19

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical | ⚡ Quick win

Build root references wrong OCP version.

The build_root.image_stream_tag.tag references openshift-4.19, but this configuration targets OCP 4.15 (as indicated by the variant ocp-4.15 and all release definitions). This mismatch will cause builds to use the wrong base environment.

🔧 Proposed fix
 build_root:
   image_stream_tag:
     name: release
     namespace: openshift
-    tag: rhel-9-release-golang-1.23-openshift-4.19
+    tag: rhel-9-release-golang-1.23-openshift-4.15
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
build_root:
image_stream_tag:
name: release
namespace: openshift
tag: rhel-9-release-golang-1.23-openshift-4.19
build_root:
image_stream_tag:
name: release
namespace: openshift
tag: rhel-9-release-golang-1.23-openshift-4.15
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.15.yaml`
around lines 1 - 5, The build root tag is pointing to the wrong OCP version:
update build_root.image_stream_tag.tag (the tag value currently contains
"openshift-4.19") to the matching OCP 4.15 release tag (replace "openshift-4.19"
with "openshift-4.15" in the tag string) so the build uses the correct base
environment for the ocp-4.15 variant.

@Anna-Koudelkova

Copy link
Copy Markdown
Contributor Author

/pj-rehearse periodic-ci-ComplianceAsCode-compliance-operator-master-ocp-4.22-e2e-aws-parallel-f14

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@Anna-Koudelkova: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@Anna-Koudelkova
Anna-Koudelkova force-pushed the periodic_CaC_co branch 2 times, most recently from f080e9c to b7245e0 Compare June 9, 2026 19:55

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.17.yaml (1)

22-26: 💤 Low value

Consider aligning global resource default with actual test usage.

The global resource default specifies cpu: 200m (line 25), but all seven test jobs override this to cpu: 100m in their test step resource requests. For consistency and clarity, consider setting the global default to match the actual usage pattern.

♻️ Proposed adjustment
 resources:
   '*':
     requests:
-      cpu: 200m
+      cpu: 100m
       memory: 400Mi

Then the per-test overrides at lines 45-47, 66-68, 87-89, 108-110, 129-131, 150-152, and 171-173 could be removed if 100m is truly the universal default.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.17.yaml`
around lines 22 - 26, The global resource default under the resources:'*' block
currently sets cpu: 200m but all seven test jobs override their test step
requests to cpu: 100m; change the global default cpu to 100m to align with
actual usage (resources:'*' -> cpu: 100m) and then remove the duplicated
per-test resource overrides in each test job (the test step resource requests
that set cpu: 100m) so the jobs inherit the global default; look for the
resources block and the test step resource overrides in the seven test jobs and
update/remove them accordingly.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In
`@ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.17.yaml`:
- Around line 22-26: The global resource default under the resources:'*' block
currently sets cpu: 200m but all seven test jobs override their test step
requests to cpu: 100m; change the global default cpu to 100m to align with
actual usage (resources:'*' -> cpu: 100m) and then remove the duplicated
per-test resource overrides in each test job (the test step resource requests
that set cpu: 100m) so the jobs inherit the global default; look for the
resources block and the test step resource overrides in the seven test jobs and
update/remove them accordingly.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 262f5874-f767-421f-86ab-09efcf48e3f9

📥 Commits

Reviewing files that changed from the base of the PR and between 8a8e060 and f080e9c.

⛔ Files ignored due to path filters (1)
  • ci-operator/jobs/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master-periodics.yaml is excluded by !ci-operator/jobs/**
📒 Files selected for processing (11)
  • ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.12.yaml
  • ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.13.yaml
  • ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.14.yaml
  • ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.15.yaml
  • ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.16.yaml
  • ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.17.yaml
  • ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.18.yaml
  • ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.19.yaml
  • ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.20.yaml
  • ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.21.yaml
  • ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.22.yaml
✅ Files skipped from review due to trivial changes (1)
  • ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.21.yaml
🚧 Files skipped from review as they are similar to previous changes (9)
  • ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.13.yaml
  • ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.15.yaml
  • ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.19.yaml
  • ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.12.yaml
  • ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.22.yaml
  • ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.16.yaml
  • ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.14.yaml
  • ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.18.yaml
  • ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.20.yaml

@Anna-Koudelkova

Copy link
Copy Markdown
Contributor Author

/pj-rehearse periodic-ci-ComplianceAsCode-compliance-operator-master-ocp-4.22-e2e-aws-parallel-f14

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@Anna-Koudelkova: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@openshift-ci

openshift-ci Bot commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

@Anna-Koudelkova: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/rehearse/periodic-ci-ComplianceAsCode-compliance-operator-master-ocp-4.22-e2e-aws-parallel-f14 b7245e0 link unknown /pj-rehearse periodic-ci-ComplianceAsCode-compliance-operator-master-ocp-4.22-e2e-aws-parallel-f14

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@Anna-Koudelkova Anna-Koudelkova changed the title WIP CMP-4326: Add CI jobs for periodic testing of the e2e tests in CaC-compliance operator repo CMP-4326: Add CI jobs for periodic testing of the e2e tests in CaC-compliance operator repo Jul 30, 2026
@openshift-ci openshift-ci Bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jul 30, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In
`@ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.12.yaml`:
- Around line 1-5: Make the build_root.image_stream_tag configuration consistent
with each OCP variant’s matching test configuration: update
ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.12.yaml,
__ocp-4.13.yaml, __ocp-4.14.yaml, __ocp-4.15.yaml, __ocp-4.16.yaml,
__ocp-4.17.yaml, and __ocp-4.18.yaml at lines 1-5 to use the appropriate
per-variant image stream and tag instead of the shared 4.19/RHEL-9 Go 1.23 root.
Alternatively, if that shared root is intentional, update the corresponding
4.20–4.22 variants to use it as well.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Pro Plus

Run ID: 8a87dfcb-5ab5-4332-9126-b45b89c571ea

📥 Commits

Reviewing files that changed from the base of the PR and between b7245e0 and 5fc082f.

⛔ Files ignored due to path filters (1)
  • ci-operator/jobs/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master-periodics.yaml is excluded by !ci-operator/jobs/**
📒 Files selected for processing (11)
  • ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.12.yaml
  • ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.13.yaml
  • ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.14.yaml
  • ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.15.yaml
  • ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.16.yaml
  • ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.17.yaml
  • ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.18.yaml
  • ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.19.yaml
  • ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.20.yaml
  • ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.21.yaml
  • ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.22.yaml

Comment on lines +1 to +5
build_root:
image_stream_tag:
name: release
namespace: openshift
tag: rhel-9-release-golang-1.23-openshift-4.19

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# What build roots do sibling ComplianceAsCode / other repos use for these OCP versions?
fd -t f 'ComplianceAsCode-compliance-operator-master' ci-operator/config \
  --exec sh -c 'printf "%s -> " "$1"; rg -m1 -N "^\s+tag: rhel-9-release" "$1"' _ {}

# Which golang-N.NN-openshift-4.1X build root tags are actually referenced anywhere in the repo?
rg -oN 'rhel-9-release-golang-[0-9.]+-openshift-4\.1[2-8]' ci-operator/config | sort -u | head -50

Repository: openshift/release

Length of output: 9129


🏁 Script executed:

#!/bin/bash
set -euo pipefail

files=(
ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.12.yaml
ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.13.yaml
ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.14.yaml
ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.15.yaml
ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.16.yaml
ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.17.yaml
ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.18.yaml
ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.19.yaml
ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.20.yaml
ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.21.yaml
ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.22.yaml
)

echo "== build_root entries =="
for f in "${files[@]}"; do
  printf '%s -> ' "$f"
  sed -n '1,8p' "$f" | rg -n 'tag: rhel-9-release' || echo 'no matching build_root tag'
done

echo "
== module/toolchain info in repo =="
git ls-files | rg '(^go\.mod$|go\.work$|Makefile$|Dockerfile.*|\.ci-operator.yaml$|\.ci\.yaml$)' | rg 'ComplianceAsCode/compliance-operator|compliance-operator' || true
rg -n 'go 1\.\d+|toolchain |GOLANG|GOVERSION|GODEBUG|rhel-9-release-golang|openshift-4\.' ci-operator/config/ComplianceAsCode/compliance-operator -g '*.yaml' | head -100

Repository: openshift/release

Length of output: 4097


🌐 Web query:

OpenShift 4.19 ci-operator build root image_stream_tag tag rhel-9-release-golang-1.23-openshift-4.19

💡 Result:

In OpenShift 4.19, the rhel-9-release-golang-1.23-openshift-4.19 tag is used within the ci-operator configuration to define the build_root_image [1][2][3]. This tag ensures that the build environment for OpenShift components is consistent with the ART (Automated Release Team) build specifications for the 4.19 release cycle [4][5]. When defined in a component's .ci-operator.yaml file, this configuration is typically structured as follows: build_root_image: name: release namespace: openshift tag: rhel-9-release-golang-1.23-openshift-4.19 The build_root provides the necessary build-time dependencies, such as the specified Go version (1.23) and RHEL 9 base environment, required for compiling components and running tests within the OpenShift CI environment [6][7][8]. Consistent use of this tag across components helps maintain parity between the component-level builds and the central ART-managed release images [4][5][9]. Additionally, this tag is often paired with corresponding builder images in the component's Dockerfile (e.g., FROM registry.ci.openshift.org/ocp/builder:rhel-9-golang-1.23-openshift-4.19 AS builder) to ensure the build environment used during CI processes remains identical to the one used for the final productized images [1][10][2].

Citations:


Make the 4.12–4.18 build roots consistent with their test variants. These seven configs still point at the 4.19/RHEL-9 Go build root while the newer sibling variants use their own matching image streams up to 4.22. If the Go 1.23 build root is intentional, align the 4.20–4.22 variants to the same root; otherwise update each job to the build root appropriate for that OCP variant.

📍 Affects 7 files
  • ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.12.yaml#L1-L5 (this comment)
  • ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.13.yaml#L1-L5
  • ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.14.yaml#L1-L5
  • ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.15.yaml#L1-L5
  • ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.16.yaml#L1-L5
  • ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.17.yaml#L1-L5
  • ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.18.yaml#L1-L5
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.12.yaml`
around lines 1 - 5, Make the build_root.image_stream_tag configuration
consistent with each OCP variant’s matching test configuration: update
ci-operator/config/ComplianceAsCode/compliance-operator/ComplianceAsCode-compliance-operator-master__ocp-4.12.yaml,
__ocp-4.13.yaml, __ocp-4.14.yaml, __ocp-4.15.yaml, __ocp-4.16.yaml,
__ocp-4.17.yaml, and __ocp-4.18.yaml at lines 1-5 to use the appropriate
per-variant image stream and tag instead of the shared 4.19/RHEL-9 Go 1.23 root.
Alternatively, if that shared root is intentional, update the corresponding
4.20–4.22 variants to use it as well.

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

[REHEARSALNOTIFIER]
@Anna-Koudelkova: the pj-rehearse plugin accommodates running rehearsal tests for the changes in this PR. Expand 'Interacting with pj-rehearse' for usage details. The following rehearsable tests have been affected by this change:

Test name Repo Type Reason
periodic-ci-ComplianceAsCode-compliance-operator-master-ocp-4.19-e2e-aws-deployment-f28 N/A periodic Periodic changed
periodic-ci-ComplianceAsCode-compliance-operator-master-ocp-4.22-e2e-aws-tailoring-f14 N/A periodic Periodic changed
periodic-ci-ComplianceAsCode-compliance-operator-master-ocp-4.15-e2e-aws-prerelease-f60 N/A periodic Periodic changed
periodic-ci-ComplianceAsCode-compliance-operator-master-ocp-4.21-e2e-aws-scan-config-f28 N/A periodic Periodic changed
periodic-ci-ComplianceAsCode-compliance-operator-master-ocp-4.12-e2e-aws-cel-f60 N/A periodic Periodic changed
periodic-ci-ComplianceAsCode-compliance-operator-master-ocp-4.19-e2e-aws-cel-f28 N/A periodic Periodic changed
periodic-ci-ComplianceAsCode-compliance-operator-master-ocp-4.17-e2e-aws-tailoring-f28 N/A periodic Periodic changed
periodic-ci-ComplianceAsCode-compliance-operator-master-ocp-4.13-e2e-aws-prerelease-f60 N/A periodic Periodic changed
periodic-ci-ComplianceAsCode-compliance-operator-master-ocp-4.21-e2e-aws-tailoring-f28 N/A periodic Periodic changed
periodic-ci-ComplianceAsCode-compliance-operator-master-ocp-4.15-e2e-aws-tailoring-f60 N/A periodic Periodic changed
periodic-ci-ComplianceAsCode-compliance-operator-master-ocp-4.17-e2e-aws-deployment-f28 N/A periodic Periodic changed
periodic-ci-ComplianceAsCode-compliance-operator-master-ocp-4.13-e2e-aws-parsing-f60 N/A periodic Periodic changed
periodic-ci-ComplianceAsCode-compliance-operator-master-ocp-4.13-e2e-aws-tailoring-f60 N/A periodic Periodic changed
periodic-ci-ComplianceAsCode-compliance-operator-master-ocp-4.14-e2e-aws-cel-f60 N/A periodic Periodic changed
periodic-ci-ComplianceAsCode-compliance-operator-master-ocp-4.19-e2e-aws-prerelease-f28 N/A periodic Periodic changed
periodic-ci-ComplianceAsCode-compliance-operator-master-ocp-4.15-e2e-aws-deployment-f60 N/A periodic Periodic changed
periodic-ci-ComplianceAsCode-compliance-operator-master-ocp-4.22-e2e-aws-prerelease-f14 N/A periodic Periodic changed
periodic-ci-ComplianceAsCode-compliance-operator-master-ocp-4.14-e2e-aws-prerelease-f60 N/A periodic Periodic changed
periodic-ci-ComplianceAsCode-compliance-operator-master-ocp-4.19-e2e-aws-scan-config-f28 N/A periodic Periodic changed
periodic-ci-ComplianceAsCode-compliance-operator-master-ocp-4.18-e2e-aws-prerelease-f14 N/A periodic Periodic changed
periodic-ci-ComplianceAsCode-compliance-operator-master-ocp-4.22-e2e-aws-cel-f14 N/A periodic Periodic changed
periodic-ci-ComplianceAsCode-compliance-operator-master-ocp-4.12-e2e-aws-deployment-f60 N/A periodic Periodic changed
periodic-ci-ComplianceAsCode-compliance-operator-master-ocp-4.22-e2e-aws-parsing-f14 N/A periodic Periodic changed
periodic-ci-ComplianceAsCode-compliance-operator-master-ocp-4.21-e2e-aws-deployment-f28 N/A periodic Periodic changed
periodic-ci-ComplianceAsCode-compliance-operator-master-ocp-4.14-e2e-aws-scan-config-f60 N/A periodic Periodic changed

A total of 66 jobs have been affected by this change. The above listing is non-exhaustive and limited to 25 jobs.

A full list of affected jobs can be found here

Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse list to get an up-to-date list of affected jobs
Comment: /pj-rehearse abort to abort all active rehearsals
Comment: /pj-rehearse network-access-allowed to allow rehearsals of tests that have the restrict_network_access field set to false. This must be executed by an openshift org member who is not the PR author

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants