INTEROP-9384: Replace OCS acceptance tests with ODF health check - #82999
INTEROP-9384: Replace OCS acceptance tests with ODF health check#82999amp-rh wants to merge 9 commits into
Conversation
|
@amp-rh: This pull request references INTEROP-9384 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "5.0.0" version, but no target version was set. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
openshift-ci-robot
added
the
jira/valid-reference
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughThe change adds a seven-point ODF health gate, registers it as an interop step, and replaces the previous OCS test in AWS and vSphere workflows for OCP 4.22 and 5.0. ChangesODF health interoperability
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant InteropWorkflow
participant interop-opp-odf-health
participant KubernetesAPI
participant NooBaaS3
participant JUnitXML
InteropWorkflow->>interop-opp-odf-health: start health checks
interop-opp-odf-health->>KubernetesAPI: query ODF and storage resources
interop-opp-odf-health->>NooBaaS3: run put, get, and delete operations
interop-opp-odf-health->>JUnitXML: write check results
interop-opp-odf-health-->>InteropWorkflow: return aggregate status
Possibly related PRs
Suggested labels: Important Pre-merge checks failedPlease resolve all errors before merging. Addressing warnings is optional. ❌ Failed checks (1 error, 1 warning, 1 inconclusive)
✅ Passed checks (12 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: amp-rh The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@ci-operator/step-registry/interop/opp/odf-health/interop-opp-odf-health-commands.sh`:
- Around line 300-353: Update the S3 check around the generated bucket variable
and embedded oc run script to create bucket before the put, use the generated
bucket instead of first.bucket in every put/get/delete command, and clean up by
deleting both the test object and bucket. Ensure the generated bucket value is
passed into and consistently used by the active script.
- Line 2: Change the script’s initial shell options from set -euxo pipefail to
set -euo pipefail, and update the NooBaa credential handling around the oc run
command to disable tracing before credentials are read or decoded and re-enable
it only after the temporary-pod command completes.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository YAML (base), Central YAML (inherited)
Review profile: CHILL
Plan: Pro Plus
Run ID: 2d32645f-d32d-472a-8dfc-5fc809c16976
📒 Files selected for processing (6)
ci-operator/config/stolostron/policy-collection/stolostron-policy-collection-main__ocp4.22.yamlci-operator/config/stolostron/policy-collection/stolostron-policy-collection-main__ocp5.0.yamlci-operator/step-registry/interop/opp/odf-health/OWNERSci-operator/step-registry/interop/opp/odf-health/interop-opp-odf-health-commands.shci-operator/step-registry/interop/opp/odf-health/interop-opp-odf-health-ref.metadata.jsonci-operator/step-registry/interop/opp/odf-health/interop-opp-odf-health-ref.yaml
openshift-ci
Bot
added
the
needs-rebase
|
@amp-rh, Interacting with pj-rehearseComment: Once you are satisfied with the results of the rehearsals, comment: |
There was a problem hiding this comment.
🧹 Nitpick comments (1)
ci-operator/step-registry/interop/opp/odf-health/interop-opp-odf-health-commands.sh (1)
297-337: 🔒 Security & Privacy | 🔵 Trivial | ⚡ Quick winKeep tracing disabled by default.
Add a comment before
set +xto identify the NooBaa credential boundary. Remove the unconditionalset -xat Line 337. Set the script default toset -euo pipefailunless a non-sensitive diagnostic section requires tracing.As per coding guidelines, “default to
set -euo pipefailwithout-x” and “add clear comments when disabling tracing.”🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@ci-operator/step-registry/interop/opp/odf-health/interop-opp-odf-health-commands.sh` around lines 297 - 337, Update the surrounding script to establish set -euo pipefail as the default without tracing, add a clear comment immediately before set +x identifying the NooBaa credential boundary, and remove the unconditional set -x after the S3 check. Keep tracing disabled unless an explicitly identified non-sensitive diagnostic section requires it.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In
`@ci-operator/step-registry/interop/opp/odf-health/interop-opp-odf-health-commands.sh`:
- Around line 297-337: Update the surrounding script to establish set -euo
pipefail as the default without tracing, add a clear comment immediately before
set +x identifying the NooBaa credential boundary, and remove the unconditional
set -x after the S3 check. Keep tracing disabled unless an explicitly identified
non-sensitive diagnostic section requires it.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository YAML (base), Central YAML (inherited)
Review profile: CHILL
Plan: Pro Plus
Run ID: f9f6c27a-f3be-4195-b4d8-00f15d3d8af0
📒 Files selected for processing (5)
ci-operator/config/stolostron/policy-collection/stolostron-policy-collection-main__ocp4.22.yamlci-operator/config/stolostron/policy-collection/stolostron-policy-collection-main__ocp5.0.yamlci-operator/step-registry/interop/opp/odf-health/interop-opp-odf-health-commands.shci-operator/step-registry/interop/opp/odf-health/interop-opp-odf-health-ref.metadata.jsonci-operator/step-registry/interop/opp/odf-health/interop-opp-odf-health-ref.yaml
💤 Files with no reviewable changes (2)
- ci-operator/config/stolostron/policy-collection/stolostron-policy-collection-main__ocp4.22.yaml
- ci-operator/config/stolostron/policy-collection/stolostron-policy-collection-main__ocp5.0.yaml
🚧 Files skipped from review as they are similar to previous changes (2)
- ci-operator/step-registry/interop/opp/odf-health/interop-opp-odf-health-ref.metadata.json
- ci-operator/step-registry/interop/opp/odf-health/interop-opp-odf-health-ref.yaml
4956c58 to
5b2c555
Compare
|
Note GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer. |
openshift-ci
Bot
removed
the
needs-rebase
5b2c555 to
46dec17
Compare
|
Note GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer. |
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (1)
ci-operator/step-registry/interop/opp/odf-health/interop-opp-odf-health-commands.sh (1)
2-2: 🔒 Security & Privacy | 🟠 Major | ⚡ Quick winDisable xtrace by default.
Line 2 enables global xtrace although this script reads NooBaa admin credentials later. Start with
set -euo pipefail. Enable xtrace only for short, non-sensitive diagnostic scopes.As per coding guidelines, step-registry command scripts must “default to
set -euo pipefailwithout-x.”🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@ci-operator/step-registry/interop/opp/odf-health/interop-opp-odf-health-commands.sh` at line 2, Update the script’s top-level shell options from `set -euxo pipefail` to `set -euo pipefail`, leaving xtrace disabled by default; only enable it in narrowly scoped diagnostic sections that do not expose NooBaa admin credentials.Sources: Coding guidelines, Learnings
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@ci-operator/step-registry/interop/opp/odf-health/interop-opp-odf-health-commands.sh`:
- Around line 315-331: Update the `oc run` invocation in the S3 check to use a
vetted pinned AWS CLI image and inject the existing NooBaa credentials via
`secretKeyRef` or `envFrom`. Remove the `AWS_ACCESS_KEY_ID` and
`AWS_SECRET_ACCESS_KEY` interpolations from the heredoc and parent shell, while
preserving the existing S3 commands and validation flow.
---
Duplicate comments:
In
`@ci-operator/step-registry/interop/opp/odf-health/interop-opp-odf-health-commands.sh`:
- Line 2: Update the script’s top-level shell options from `set -euxo pipefail`
to `set -euo pipefail`, leaving xtrace disabled by default; only enable it in
narrowly scoped diagnostic sections that do not expose NooBaa admin credentials.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository YAML (base), Central YAML (inherited)
Review profile: CHILL
Plan: Pro Plus
Run ID: b3369447-fc35-433f-9dbb-2a5b3c400941
📒 Files selected for processing (6)
ci-operator/config/stolostron/policy-collection/stolostron-policy-collection-main__ocp4.22.yamlci-operator/config/stolostron/policy-collection/stolostron-policy-collection-main__ocp5.0.yamlci-operator/step-registry/interop/opp/odf-health/OWNERSci-operator/step-registry/interop/opp/odf-health/interop-opp-odf-health-commands.shci-operator/step-registry/interop/opp/odf-health/interop-opp-odf-health-ref.metadata.jsonci-operator/step-registry/interop/opp/odf-health/interop-opp-odf-health-ref.yaml
🚧 Files skipped from review as they are similar to previous changes (4)
- ci-operator/step-registry/interop/opp/odf-health/interop-opp-odf-health-ref.metadata.json
- ci-operator/step-registry/interop/opp/odf-health/OWNERS
- ci-operator/config/stolostron/policy-collection/stolostron-policy-collection-main__ocp5.0.yaml
- ci-operator/step-registry/interop/opp/odf-health/interop-opp-odf-health-ref.yaml
ODF Health Gate Review — Domain AnalysisReviewed the 7-point health gate approach, ODF health signal coverage, NooBaa S3 pattern, and risks from removing the OCS acceptance suite. Overall this is a well-motivated change — the old step ran 28 single-product ODF tests with zero cross-product coverage and even deleted the ACM subscription before executing, fundamentally undermining its interop purpose. 1. Is the 7-point gate sufficient for cross-product interop?Yes, for its intended purpose. The gate validates that ODF is healthy and functional within the OPP stack, which is the correct scope for an interop pipeline. The old acceptance suite validated feature correctness and robustness, but none of that was cross-product — it was single-product ODF QE validation running in the wrong pipeline. The 7 checks cover the key readiness signals documented in ODF CI patterns:
One gap worth considering: The 2. Missing ODF health signalsThree signals I'd recommend adding or considering: a) CephFS PVC validation (medium priority)Check 5 only tests # Add after the ceph-rbd PVC test
storageClassName: ocs-storagecluster-cephfs
accessModes:
- ReadWriteManyb) OSD "up + in" state (low priority, nice-to-have)The CephCluster oc get cephcluster -n openshift-storage -o json | jq '.items[0].status.ceph.details'This is partially covered by Check 7 (Ceph health detail), but if all OSDs are up and health is HEALTH_WARN for a non-OSD reason (e.g., clock skew), you'd want to know. c) StorageCluster
|
d7a856a to
de775fe
Compare
|
Thanks for the thorough review. Addressing each finding: Already implemented (may have been missed in review):
Deferred (scope creep for this PR):
|
|
Note GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer. |
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (1)
ci-operator/step-registry/interop/opp/odf-health/interop-opp-odf-health-commands.sh (1)
377-380: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winKeep AWS CLI stderr for diagnosis.
Each
awscall discards stderr. When the S3 cycle fails, the pod logs show nothing, and the JUnit message stays generic. Keep stderr so the artifacts explain the failure.♻️ Proposed change
- echo "\${TEST_DATA}" | aws --endpoint-url "\${S3_ENDPOINT}" --no-verify-ssl s3 cp - "s3://\${BUCKET_NAME}/\${TEST_KEY}" 2>/dev/null && \ - RETRIEVED=\$(aws --endpoint-url "\${S3_ENDPOINT}" --no-verify-ssl s3 cp "s3://\${BUCKET_NAME}/\${TEST_KEY}" - 2>/dev/null) && \ - aws --endpoint-url "\${S3_ENDPOINT}" --no-verify-ssl s3 rm "s3://\${BUCKET_NAME}/\${TEST_KEY}" 2>/dev/null && \ + echo "\${TEST_DATA}" | aws --endpoint-url "\${S3_ENDPOINT}" --no-verify-ssl s3 cp - "s3://\${BUCKET_NAME}/\${TEST_KEY}" && \ + RETRIEVED=\$(aws --endpoint-url "\${S3_ENDPOINT}" --no-verify-ssl s3 cp "s3://\${BUCKET_NAME}/\${TEST_KEY}" -) && \ + aws --endpoint-url "\${S3_ENDPOINT}" --no-verify-ssl s3 rm "s3://\${BUCKET_NAME}/\${TEST_KEY}" && \ if [ "\${RETRIEVED}" = "\${TEST_DATA}" ]; then echo "S3_CHECK_PASS"; else echo "S3_CHECK_FAIL: data mismatch"; fi🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@ci-operator/step-registry/interop/opp/odf-health/interop-opp-odf-health-commands.sh` around lines 377 - 380, Update the S3 validation command chain in the interop health check to stop redirecting stderr from each aws invocation to /dev/null. Preserve the existing upload, retrieval, cleanup, and data-comparison behavior while allowing AWS CLI diagnostics to appear in the pod logs and test artifacts.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@ci-operator/step-registry/interop/opp/odf-health/interop-opp-odf-health-commands.sh`:
- Around line 328-334: In the OBC credential setup block, validate that
bucketName returned by the bucket lookup is non-empty before continuing to S3
checks. On an empty value, record a JUnit failure with a clear
bucket-name-specific message and return or otherwise stop the current check,
while preserving the existing secretRef fallback for valid bucket names.
- Around line 381-391: Update the pod execution flow around the manifest’s
activeDeadlineSeconds and the oc wait in the S3 health check: define a separate
NOOBAA_S3_POD_STARTUP_TIMEOUT defaulted to 120 seconds, use it for
activeDeadlineSeconds, and wait for the pod’s terminal completion phase rather
than condition=Ready with the S3 operation timeout. Preserve collecting oc logs
after completion or timeout.
---
Nitpick comments:
In
`@ci-operator/step-registry/interop/opp/odf-health/interop-opp-odf-health-commands.sh`:
- Around line 377-380: Update the S3 validation command chain in the interop
health check to stop redirecting stderr from each aws invocation to /dev/null.
Preserve the existing upload, retrieval, cleanup, and data-comparison behavior
while allowing AWS CLI diagnostics to appear in the pod logs and test artifacts.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository YAML (base), Central YAML (inherited)
Review profile: CHILL
Plan: Pro Plus
Run ID: 12be18b9-743f-4d44-8b9a-e714169be362
📒 Files selected for processing (6)
ci-operator/config/stolostron/policy-collection/stolostron-policy-collection-main__ocp4.22.yamlci-operator/config/stolostron/policy-collection/stolostron-policy-collection-main__ocp5.0.yamlci-operator/step-registry/interop/opp/odf-health/OWNERSci-operator/step-registry/interop/opp/odf-health/interop-opp-odf-health-commands.shci-operator/step-registry/interop/opp/odf-health/interop-opp-odf-health-ref.metadata.jsonci-operator/step-registry/interop/opp/odf-health/interop-opp-odf-health-ref.yaml
🚧 Files skipped from review as they are similar to previous changes (5)
- ci-operator/step-registry/interop/opp/odf-health/interop-opp-odf-health-ref.metadata.json
- ci-operator/step-registry/interop/opp/odf-health/OWNERS
- ci-operator/step-registry/interop/opp/odf-health/interop-opp-odf-health-ref.yaml
- ci-operator/config/stolostron/policy-collection/stolostron-policy-collection-main__ocp5.0.yaml
- ci-operator/config/stolostron/policy-collection/stolostron-policy-collection-main__ocp4.22.yaml
|
Note GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer. |
|
/pj-rehearse periodic-ci-stolostron-policy-collection-main-ocp4.22-interop-opp-vsphere |
|
@amp-rh: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
There was a problem hiding this comment.
♻️ Duplicate comments (1)
ci-operator/step-registry/interop/opp/odf-health/interop-opp-odf-health-commands.sh (1)
386-396: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick winSeparate the image pull budget from the S3 operation timeout.
Line 391 now waits for the
Succeededphase withpodWait, which resolves half of the earlier concern. Line 386 still setsactiveDeadlineSecondstoNOOBAA_S3_TIMEOUT(30 by default).activeDeadlineSecondsstarts when the pod is scheduled, so it includes the pull ofamazon/aws-cli:2.22.35from Docker Hub. If the pull takes more than 30 seconds, the kubelet terminates the pod withDeadlineExceededbeforeawsruns. The logs are then empty and the gate reports "NooBaa S3 functional check failed" while NooBaa is healthy.Give the pod the same budget that
oc waituses.🔧 Proposed fix
- activeDeadlineSeconds: ${NOOBAA_S3_TIMEOUT} + activeDeadlineSeconds: $(( NOOBAA_S3_TIMEOUT + 60 )) EOF ) typeset s3Result="" typeset -i podWait=$(( NOOBAA_S3_TIMEOUT + 60 )) if echo "${podManifest}" | oc apply -f -; then if ! oc wait pod "${podName}" -n "${ODF_NAMESPACE}" \ --for=jsonpath='{.status.phase}'=Succeeded \ --timeout="${podWait}s" 2>/dev/null; then : "Pod did not succeed within ${podWait}s, checking logs anyway" + oc describe pod "${podName}" -n "${ODF_NAMESPACE}" || true fi🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@ci-operator/step-registry/interop/opp/odf-health/interop-opp-odf-health-commands.sh` around lines 386 - 396, Update the pod manifest’s activeDeadlineSeconds value in the NooBaa S3 health-check flow to use the same podWait budget passed to oc wait, rather than NOOBAA_S3_TIMEOUT. Ensure podWait is calculated before constructing the manifest, preserving the existing S3 operation timeout for the AWS command itself.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Duplicate comments:
In
`@ci-operator/step-registry/interop/opp/odf-health/interop-opp-odf-health-commands.sh`:
- Around line 386-396: Update the pod manifest’s activeDeadlineSeconds value in
the NooBaa S3 health-check flow to use the same podWait budget passed to oc
wait, rather than NOOBAA_S3_TIMEOUT. Ensure podWait is calculated before
constructing the manifest, preserving the existing S3 operation timeout for the
AWS command itself.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository YAML (base), Central YAML (inherited)
Review profile: CHILL
Plan: Pro Plus
Run ID: 2ef28719-14bf-4360-ae03-92ec9b0a5f56
📒 Files selected for processing (6)
ci-operator/config/stolostron/policy-collection/stolostron-policy-collection-main__ocp4.22.yamlci-operator/config/stolostron/policy-collection/stolostron-policy-collection-main__ocp5.0.yamlci-operator/step-registry/interop/opp/odf-health/OWNERSci-operator/step-registry/interop/opp/odf-health/interop-opp-odf-health-commands.shci-operator/step-registry/interop/opp/odf-health/interop-opp-odf-health-ref.metadata.jsonci-operator/step-registry/interop/opp/odf-health/interop-opp-odf-health-ref.yaml
🚧 Files skipped from review as they are similar to previous changes (5)
- ci-operator/step-registry/interop/opp/odf-health/OWNERS
- ci-operator/config/stolostron/policy-collection/stolostron-policy-collection-main__ocp4.22.yaml
- ci-operator/step-registry/interop/opp/odf-health/interop-opp-odf-health-ref.metadata.json
- ci-operator/step-registry/interop/opp/odf-health/interop-opp-odf-health-ref.yaml
- ci-operator/config/stolostron/policy-collection/stolostron-policy-collection-main__ocp5.0.yaml
|
/pj-rehearse periodic-ci-stolostron-policy-collection-main-ocp4.22-interop-opp-vsphere |
|
@amp-rh: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
@amp-rh: The following test failed, say
Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
2a1c962 to
fb618be
Compare
openshift-ci
Bot
added
the
needs-rebase
The interop-tests-ocs-tests step runs 28 single-product ODF acceptance tests after deliberately deleting the ACM subscription, making them fundamentally misaligned with interop testing. The step has a 3h timeout, depends on an external ocs-ci image, and the NooBaa endpoint timeout (OCSQE-5148) has been unresponsive for 16+ days. Replace with a lightweight 7-point ODF health gate (~2 min): 1. ODF Operator CSV in Succeeded phase 2. StorageCluster phase == Ready 3. CephCluster health == HEALTH_OK or HEALTH_WARN 4. Default StorageClasses available (ceph-rbd, cephfs) 5. PVC provisionable (create, bind, delete) 6. NooBaa system Ready + S3 put/get/delete functional check 7. Ceph overall health detail The NooBaa S3 functional check (check 6) is critical because Quay has a hard dependency on NooBaa for image blob storage. If NooBaa accepts connections but cannot serve object operations, Quay will silently fail on push/pull. Produces JUnit XML for Prow/Sippy/TestGrid. No external image dependency. Does NOT delete the ACM subscription (keeps stack intact for true interop).
- Remove ODF_VERSION_MAJOR_MINOR and DISABLE_ENVIRONMENT_CHECKER env overrides (only declared by removed interop-tests-ocs-tests step) - Remove unused ocs-ci-tests base_image (required by validateUnused) - Remove dead code: bucket, s3ok, s3Script variables (SC2034) - Fix SC2155: split testData declaration and assignment - Guard credentials with set +x/set -x to prevent log exposure - Add true terminators to all functions (mpex best practice) - Bump step timeout from 5m to 8m (PVC bind + cold image pull buffer) - Fix metadata.json trailing newline
All function declarations now use `function FuncName () {` form
per MPEX Integrity Engineering Best Practices.
Address CodeRabbit security finding: credentials were interpolated into the pod command spec. Now uses envFrom with the existing NooBaa admin secret. Pins aws-cli to 2.22.35 instead of :latest.
Check 5 now validates both ceph-rbd and cephfs provisioning. Check 6 creates an ObjectBucketClaim instead of using the legacy first.bucket, avoiding future breakage and key collisions.
- Use `set -eux -o pipefail` (separated -o flag) - Replace &>/dev/null with `-o name 2>/dev/null` on sc existence check - Remove stderr suppression on oc apply/wait where error context aids debugging
Address CodeRabbit review findings: - Guard against empty OBC bucket name before launching S3 pod - Wait for pod phase=Succeeded instead of condition=Ready - Add 60s image pull budget on top of S3 timeout
- Rename JUnit testsuite to lp-interop--ODF for Sippy allowlist - Declare all loop iterator variables with typeset (mpitt) - Add env declarations to ref.yaml for discoverability (mpitt) - Add securityContext to S3 check pod (runAsNonRoot, no escalation)
fb618be to
a87ebfe
Compare
openshift-ci
Bot
removed
the
needs-rebase
- Remove xtrace (-x) to avoid logging S3 endpoint in CI output - Add readOnlyRootFilesystem, drop ALL capabilities, resource limits, and emptyDir volume mount to the s3check Pod manifest
|
[REHEARSALNOTIFIER]
Interacting with pj-rehearseComment: Once you are satisfied with the results of the rehearsals, comment: |
3 participants
TL;DR
Replace
interop-tests-ocs-testswith a lightweightinterop-opp-odf-healthgate in OPP interop pipelines (OCP 4.22 + 5.0). Cuts ODF validation from 28 single-product tests (3h timeout, external image dependency) to a 7-point health check (~2 min, self-contained).What changed
interop-opp-odf-healthvalidates ODF health via 7 gates (CSV phase, StorageCluster readiness, CephCluster health, StorageClass availability, PVC provisioning, NooBaa S3 functional check, Ceph overall health)interop-tests-ocs-tests→interop-opp-odf-healthocs-ci-testsbase image,MAP_TESTS,ODF_VERSION_MAJOR_MINOR,DISABLE_ENVIRONMENT_CHECKER,DR__RP__CR_COMP_NAMEenv varsftan@redhat.com→mpruitt@redhat.com(OCP 4.22 configs; 5.0 was already correct)Review guide
Core file (review this first)
ci-operator/step-registry/interop/opp/odf-health/interop-opp-odf-health-commands.shMechanical / config files
stolostron-policy-collection-main__ocp4.22.yamlstolostron-policy-collection-main__ocp5.0.yamlinterop-opp-odf-health-ref.yamlinterop-opp-odf-health-ref.metadata.jsonOWNERScspi-qe-ocp-lpteam ownershipRisk areas
amazon/aws-cli:2.22.35to do put/get/delete against the NooBaa S3 endpoint. If the pod image pull or NooBaa endpoint is slow, this could extend runtime. Mitigated byNOOBAA_S3_TIMEOUT(default 30s) and pod cleanup in all exit paths.HEALTH_WARNis treated as a pass (with a message); onlyHEALTH_ERRfails. This matches ODF QE convention, but a cluster inHEALTH_WARNduring tests could mask issues.ocs-ciacceptance tests. ODF single-product coverage is owned by ODF QE; this PR scopes OPP's gate to interop-relevant health validation only.Testing
shellcheck(clean)make updaterun to regenerate derived filesAffected jobs
periodic-ci-stolostron-policy-collection-main-ocp4.22-interop-opp-awsperiodic-ci-stolostron-policy-collection-main-ocp4.22-interop-opp-vsphereperiodic-ci-stolostron-policy-collection-main-ocp5.0-interop-opp-awsperiodic-ci-stolostron-policy-collection-main-ocp5.0-interop-opp-vsphereFollow-up (separate PRs)
cspi-utilsfirewatch config to reference new step nameopenshift-eng/ci-test-mappingforlp-interop--ODFmappingContext
Problem: The OCS acceptance step runs 28 single-product ODF tests after deleting the ACM subscription, making them fundamentally misaligned with interop testing. The step has a 3h timeout, depends on an external
ocs-ciimage, and the NooBaa endpoint timeout (OCSQE-5148) has been unresponsive for 16+ days (5 outreach attempts, 0 replies).Solution: Lightweight 7-point ODF health gate (~2 min) that validates ODF is healthy and functional without running single-product tests. Preserves the full OPP stack for true cross-product validation.
/cc @cspi-qe-ocp-lp
fixes INTEROP-9384