CNTRLPLANE-3000:test/e2e: migrate time-based-ca-rotation test for OTE compatibility#308
Conversation
|
Important Review skippedAuto reviews are limited based on label configuration. 🚫 Review skipped — only excluded labels are configured. (1)
Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughAdds a TB (testing.TB)-based time-based CA rotation end-to-end test flow in Changes
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes ✨ Finishing Touches🧪 Generate unit tests (beta)
📝 Coding Plan
Comment |
openshift-ci
Bot
added
the
approved
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Fix all issues with AI agents
In `@test/e2e/e2e.go`:
- Around line 1334-1353: The current checkRotationGinkgo skips validation for
Ginkgo because util.CheckRotation requires *testing.T (it uses t.Run); to fix,
either refactor util.CheckRotation to extract its core validation into a new
helper that accepts testing.TB (removing subtest t.Run usage) and call that from
both util.CheckRotation and checkRotationGinkgo, or implement the rotation
verification directly inside checkRotationGinkgo (replicating
util.CheckRotation's core checks but without using t.Run/subtests) so Ginkgo
gets full validation; locate the logic around checkRotationGinkgo and
util.CheckRotation and move shared verification into a testing-TB-compatible
function or inline the checks in checkRotationGinkgo.
🧹 Nitpick comments (1)
test/e2e/e2e.go (1)
1418-1418: Discarding the returned secret from pollForCARotationGinkgo.The return value of
pollForCARotationGinkgois being discarded. While this mirrors the pattern intriggerTimeBasedRotationin e2e_test.go, consider whether the returned secret should be used for additional validation or if the function should be simplified to not return a value if it's not needed.
openshift-ci
Bot
added
the
do-not-merge/work-in-progress
wangke19
force-pushed
the
ote-migrate-time-based-ca-rotation
branch
2 times, most recently
from
7357840 to
eac725d
Compare
openshift-ci
Bot
removed
the
do-not-merge/work-in-progress
|
/test e2e-aws-operator-serial-ote |
wangke19
force-pushed
the
ote-migrate-time-based-ca-rotation
branch
from
eac725d to
a02415d
Compare
openshift-merge-robot
added
the
needs-rebase
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (1)
test/e2e/e2e.go (1)
1422-1428:⚠️ Potential issue | 🟠 MajorOTE still skips the trust-matrix part of this test.
When
tcomes fromg.GinkgoTB(), this helper only logs and returns, so the migrated OTE path never runsutil.CheckRotation's pre/server-rotated/client-refreshed cases fromtest/util/rotate.go. Please move that matrix into atesting.TB-compatible helper, or inline it here, so both runners execute the same assertions.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@test/e2e/e2e.go` around lines 1422 - 1428, The Ginkgo path in checkRotationGinkgo currently skips the trust-matrix assertions; move the trust-matrix logic from util.CheckRotation (test/util/rotate.go) into a testing.TB-compatible helper (or inline the matrix into checkRotationGinkgo) so both code paths execute the same pre/server-rotated/client-refreshed checks; update checkRotationGinkgo to call that new TB-compatible helper when t is a *testing.T (or when t comes from g.GinkgoTB()) and remove the early-return log path so the migrated OTE runner exercises the same assertions as util.CheckRotation.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@test/e2e/e2e.go`:
- Around line 1442-1444: The test captures oldCACertPEM and oldCAKeyPEM from the
secret but then injects a short-lived CA via triggerTimeBasedRotationGinkgo
before calling pollForCARotationGinkgo, so those old* variables are unused and
cause the wait to succeed on the test-injected CA; remove the dead assignments
oldCACertPEM := secret.Data[v1.TLSCertKey] and oldCAKeyPEM :=
secret.Data[v1.TLSPrivateKeyKey] (and any other identical occurrences around the
other reported locations) and ensure pollForCARotationGinkgo is only comparing
against the operator-rotated CA rather than the test-injected bytes.
---
Duplicate comments:
In `@test/e2e/e2e.go`:
- Around line 1422-1428: The Ginkgo path in checkRotationGinkgo currently skips
the trust-matrix assertions; move the trust-matrix logic from util.CheckRotation
(test/util/rotate.go) into a testing.TB-compatible helper (or inline the matrix
into checkRotationGinkgo) so both code paths execute the same
pre/server-rotated/client-refreshed checks; update checkRotationGinkgo to call
that new TB-compatible helper when t is a *testing.T (or when t comes from
g.GinkgoTB()) and remove the early-return log path so the migrated OTE runner
exercises the same assertions as util.CheckRotation.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 6223cc03-8ef0-4bc1-8bd9-4298a63b300c
📒 Files selected for processing (2)
test/e2e/e2e.gotest/e2e/e2e_test.go
🚧 Files skipped from review as they are similar to previous changes (1)
- test/e2e/e2e_test.go
wangke19
force-pushed
the
ote-migrate-time-based-ca-rotation
branch
from
a02415d to
8a3d1ba
Compare
There was a problem hiding this comment.
♻️ Duplicate comments (1)
test/e2e/e2e.go (1)
1437-1439:⚠️ Potential issue | 🟠 MajorPoll succeeds immediately on test-injected CA, not operator-rotated CA.
After writing
renewedCACertPEM/renewedCAKeyPEMto the secret at line 1477, the poll at line 1482 compares againstoldCACertPEM/oldCAKeyPEM. Since the renewed values differ from old, the poll succeeds immediately—before the operator performs the actual rotation.The poll should wait for values different from the renewed (test-injected) CA to confirm the operator rotated it.
Proposed fix
- _ = pollForCARotationTB(t, client, oldCACertPEM, oldCAKeyPEM) + _ = pollForCARotationTB(t, client, renewedCACertPEM, renewedCAKeyPEM)After this fix,
oldCACertPEMandoldCAKeyPEMbecome unused and can be removed.Also applies to: 1482-1482
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@test/e2e/e2e.go` around lines 1437 - 1439, The poll is incorrectly comparing the secret against oldCACertPEM/oldCAKeyPEM (the pre-test values) so it succeeds immediately after you write the test-injected renewedCACertPEM/renewedCAKeyPEM; change the poll to wait for values different from renewedCACertPEM and renewedCAKeyPEM (i.e., compare against renewedCACertPEM/renewedCAKeyPEM in the polling predicate used around the poll at line ~1482) so the test only passes after the operator rotates the CA, and remove the now-unused oldCACertPEM and oldCAKeyPEM variables.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Duplicate comments:
In `@test/e2e/e2e.go`:
- Around line 1437-1439: The poll is incorrectly comparing the secret against
oldCACertPEM/oldCAKeyPEM (the pre-test values) so it succeeds immediately after
you write the test-injected renewedCACertPEM/renewedCAKeyPEM; change the poll to
wait for values different from renewedCACertPEM and renewedCAKeyPEM (i.e.,
compare against renewedCACertPEM/renewedCAKeyPEM in the polling predicate used
around the poll at line ~1482) so the test only passes after the operator
rotates the CA, and remove the now-unused oldCACertPEM and oldCAKeyPEM
variables.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 64b26335-aca3-410e-9d04-1959a8728d82
📒 Files selected for processing (2)
test/e2e/e2e.gotest/e2e/e2e_test.go
🚧 Files skipped from review as they are similar to previous changes (1)
- test/e2e/e2e_test.go
wangke19
force-pushed
the
ote-migrate-time-based-ca-rotation
branch
from
8a3d1ba to
1de8ad3
Compare
openshift-merge-robot
removed
the
needs-rebase
There was a problem hiding this comment.
♻️ Duplicate comments (2)
test/e2e/e2e.go (2)
1570-1576:⚠️ Potential issue | 🟠 MajorOTE still skips the actual rotation assertions.
These
util.CheckRotationcalls only run for*testing.T, so the newg.GinkgoTB()path stops after checking that certs and bundles changed and never exercises the pre/post-rotation trust matrix. Please move those checks into atesting.TB-compatible helper and call it unconditionally here.Verify the split by inspecting the new OTE call site, this type gate, and
util.CheckRotation. Expected result: only the legacy Go-test path can execute the trust-matrix assertions.#!/bin/bash set -euo pipefail printf '\n--- OTE call site ---\n' sed -n '121,125p' test/e2e/e2e.go printf '\n--- type gate in shared helper ---\n' sed -n '1568,1576p' test/e2e/e2e.go printf '\n--- util.CheckRotation implementation ---\n' sed -n '22,54p' test/util/rotate.go🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@test/e2e/e2e.go` around lines 1570 - 1576, The current type-guard only runs the detailed trust-matrix assertions when t is *testing.T, so the new g.GinkgoTB() path skips them; change util.CheckRotation to be testing.TB-compatible (e.g., add or refactor to util.CheckRotationTB(tb testing.TB, dns, oldCert, oldKey, oldBundle, newCert, newKey, newBundle)) or overload it to accept testing.TB, then call that TB-compatible helper unconditionally here (replace the type assertion block that checks for *testing.T and the per-host loop with a direct call to the testing.TB helper using the existing dnsName and headless names and PEM variables) so both legacy Go tests and OTE/ginkgo paths execute the pre/post-rotation trust-matrix assertions.
1592-1594:⚠️ Potential issue | 🟠 MajorWait on the operator-rotated CA, not the short-lived CA you inject.
After
ApplySecret, polling againstoldCA*can succeed as soon as the secret reflects the short-lived CA written by the test. That lets the remaining waits observe the intermediate CA instead of the operator-rotated one.Minimal fix
- // Store the old PEMs for comparison - oldCACertPEM := secret.Data[v1.TLSCertKey] - oldCAKeyPEM := secret.Data[v1.TLSPrivateKeyKey] ... - _ = pollForCARotationTB(t, client, oldCACertPEM, oldCAKeyPEM) + _ = pollForCARotationTB(t, client, renewedCACertPEM, renewedCAKeyPEM)Also applies to: 1637-1637
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@test/e2e/e2e.go` around lines 1592 - 1594, The test is saving oldCACertPEM/oldCAKeyPEM too late (after ApplySecret), so subsequent polls can match the short-lived CA you just injected; instead capture the operator-rotated CA before you call ApplySecret (i.e., GET the current secret and store its Data[v1.TLSCertKey] and Data[v1.TLSPrivateKeyKey] into oldCACertPEM/oldCAKeyPEM prior to invoking ApplySecret) and use those stored values for the polling comparisons so the waits target the operator-rotated CA rather than the injected one.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Duplicate comments:
In `@test/e2e/e2e.go`:
- Around line 1570-1576: The current type-guard only runs the detailed
trust-matrix assertions when t is *testing.T, so the new g.GinkgoTB() path skips
them; change util.CheckRotation to be testing.TB-compatible (e.g., add or
refactor to util.CheckRotationTB(tb testing.TB, dns, oldCert, oldKey, oldBundle,
newCert, newKey, newBundle)) or overload it to accept testing.TB, then call that
TB-compatible helper unconditionally here (replace the type assertion block that
checks for *testing.T and the per-host loop with a direct call to the testing.TB
helper using the existing dnsName and headless names and PEM variables) so both
legacy Go tests and OTE/ginkgo paths execute the pre/post-rotation trust-matrix
assertions.
- Around line 1592-1594: The test is saving oldCACertPEM/oldCAKeyPEM too late
(after ApplySecret), so subsequent polls can match the short-lived CA you just
injected; instead capture the operator-rotated CA before you call ApplySecret
(i.e., GET the current secret and store its Data[v1.TLSCertKey] and
Data[v1.TLSPrivateKeyKey] into oldCACertPEM/oldCAKeyPEM prior to invoking
ApplySecret) and use those stored values for the polling comparisons so the
waits target the operator-rotated CA rather than the injected one.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: e90a2b1d-6805-47db-bfce-d7f81d33d0f5
📒 Files selected for processing (2)
test/e2e/e2e.gotest/e2e/e2e_test.go
wangke19
force-pushed
the
ote-migrate-time-based-ca-rotation
branch
2 times, most recently
from
d9bde45 to
fb13633
Compare
|
/retest |
2 similar comments
|
/retest |
|
/retest |
|
@coderabbitai review the PR |
|
✅ Actions performedReview triggered.
|
There was a problem hiding this comment.
♻️ Duplicate comments (1)
test/e2e/e2e.go (1)
1582-1588:⚠️ Potential issue | 🟠 MajorRotation trust validation is still skipped in the OTE/Ginkgo path.
At Line 1582, validation runs only when
tis*testing.T. When invoked fromg.GinkgoTB(), this block is skipped, soutil.CheckRotationnever executes in the migrated path. That reduces effective coverage of thetime-based-ca-rotationmigration. Please run an equivalent TB-compatible rotation check unconditionally for both runners.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@test/e2e/e2e.go` around lines 1582 - 1588, The code only calls util.CheckRotation when t is *testing.T, skipping it for the Ginkgo path; fix by obtaining a testing.TB instance unconditionally (e.g., declare var tb testing.TB; if testT, ok := t.(*testing.T); ok { tb = testT } else { tb = g.GinkgoTB() }) and replace the util.CheckRotation calls to use tb so rotation validation runs for both runners; keep the same dnsName construction and headless loop with tb passed into util.CheckRotation.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Duplicate comments:
In `@test/e2e/e2e.go`:
- Around line 1582-1588: The code only calls util.CheckRotation when t is
*testing.T, skipping it for the Ginkgo path; fix by obtaining a testing.TB
instance unconditionally (e.g., declare var tb testing.TB; if testT, ok :=
t.(*testing.T); ok { tb = testT } else { tb = g.GinkgoTB() }) and replace the
util.CheckRotation calls to use tb so rotation validation runs for both runners;
keep the same dnsName construction and headless loop with tb passed into
util.CheckRotation.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: efcf8ad5-3d88-4001-88e0-78df1f864ede
📒 Files selected for processing (2)
test/e2e/e2e.gotest/e2e/e2e_test.go
|
/retest |
|
/retest |
Move time-based CA rotation test to OTE compatibility layer and the serial-disruptive suite with control plane stabilization wait. Changes: - Migrate test from e2e_test.go to e2e.go for OTE compatibility - Add [Disruptive][Timeout:20m] markers to indicate expected disruption - Add 15-minute best-effort control plane stabilization wait after rotation - Fix rotation validation to run for both *testing.T and Ginkgo (testing.TB) - Remove duplicate pollForUpdatedServingCert helper (use existing version) This prevents false-positive monitor test failures from expected kube-apiserver disruption during CA rotation.
wangke19
force-pushed
the
ote-migrate-time-based-ca-rotation
branch
from
3718ce7 to
e7f8a9d
Compare
|
@wangke19: all tests passed! Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
|
/verified by CI |
openshift-ci-robot
added
the
verified
|
@wangke19: This PR has been marked as verified by DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
wangke19
changed the title
openshift-ci-robot
added
the
jira/valid-reference
|
@wangke19: This pull request references CNTRLPLANE-3000 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the sub-task to target the "4.22.0" version, but no target version was set. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
/jira refresh |
|
@wangke19: This pull request references CNTRLPLANE-3000 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the sub-task to target the "4.22.0" version, but no target version was set. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
/lgtm |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: wangke19, zhouying7780 The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
4 participants
Migrates the
time-based-ca-rotationtest to the OTE (OpenShift Test Extended) Ginkgo framework, following the same pattern as PRs #304–#306.What changed
e2e.go: Add atime-based-ca-rotationGinkgo context and copy the required functions frome2e_test.goastesting.TB-compatible versions. Functions are verbatim copies with only mechanical changes:*testing.T→testing.TB, andserviceCAControllerNamespace/signingKeySecretNamereplaced with their equivalent constants already used ine2e.go.e2e_test.go: RemovetriggerTimeBasedRotation()(moved toe2e.go) and updatet.Run("time-based-ca-rotation")to call the sharedtestTimeBasedCARotation(t)entry point.The duplication between
e2e.goande2e_test.gois intentional and temporary — functions ine2e_test.goare not visible atgo buildtime, so verbatim TB copies ine2e.goare required. Once OTE migration is validated, thee2e_test.goversions will be removed.