test: new auth providers for e2e

[jrichter1]

just testing changes for the utils

[jrichter1]

/test e2e-helm-ocp-nightly

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[jrichter1]

/test e2e-ocp-helm-nightly

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 10:11 AM UTC · Completed 10:14 AM UTC

Commit: 4cbaab8 · View workflow run →

[fullsend-ai-review]

Review — ⛔ Changes Requested

PR: #3355 — test: new auth providers for e2e
Author: jrichter1 (member)
Scope: 1 file changed (run-e2e.sh), 1 addition, 1 deletion

Summary

This PR replaces the environment-variable-driven default for E2E_TEST_UTILS_GIT_REF with a hardcoded reference to a personal fork branch (jrichter1/rhdh-e2e-test-utils#auth-providers). The PR description says "just testing changes for the utils," confirming this is intended for local experimentation — but the PR targets main and is not marked as a draft.

Findings

🔴 High — Hardcoded personal fork breaks all E2E infrastructure

File: run-e2e.sh, line 82

The change replaces:

E2E_TEST_UTILS_GIT_REF="${E2E_TEST_UTILS_GIT_REF:-}"

with:

E2E_TEST_UTILS_GIT_REF="jrichter1/rhdh-e2e-test-utils#auth-providers"

Impact if merged:

• All E2E runs (nightly CI, PR checks, local dev via run-e2e.sh) would clone and build from a personal fork branch instead of using the published @red-hat-developer-hub/e2e-test-utils npm package.
• The environment variable override is completely removed — setting E2E_TEST_UTILS_GIT_REF externally has no effect since the script no longer reads from the environment (no ${...:-} expansion).
• The E2E_TEST_UTILS_PATH and E2E_TEST_UTILS_VERSION configuration paths (lines 78–80) become dead code: the hardcoded git ref always triggers the clone block (lines 84–90), which unconditionally overwrites E2E_TEST_UTILS_PATH.
• CI reliability becomes dependent on the availability and state of a personal fork branch that could be force-pushed, deleted, or diverge from the upstream API at any time.

Remediation: If the intent is to test changes to rhdh-e2e-test-utils, use the existing env variable mechanism locally or in a PR check:

E2E_TEST_UTILS_GIT_REF="jrichter1/rhdh-e2e-test-utils#auth-providers" ./run-e2e.sh -w <workspace>

Do not commit this hardcoded value to main. If this PR is meant purely for CI experimentation (e.g., triggering a /test run with the fork), consider marking it as a draft or adding the do-not-merge/work-in-progress label.

Verdict

request-changes — This change must not be merged to main. It hardcodes a personal fork reference into the shared E2E test runner, which would break the environment variable configuration contract and make all E2E runs dependent on an external personal repository.

---

Labels: PR modifies the E2E test runner script and appears to be a work-in-progress test change

[fullsend-ai-review]

See the review comment for full details.

[fullsend-ai-review]

[high] correctness

E2E_TEST_UTILS_GIT_REF is hardcoded to a personal fork branch (jrichter1/rhdh-e2e-test-utils#auth-providers), replacing the environment variable override pattern. If merged, all E2E runs (nightly, PR checks, local) would clone from a personal fork instead of using the published npm package. The env var override is removed, and E2E_TEST_UTILS_PATH/E2E_TEST_UTILS_VERSION settings become dead code.

Suggested fix: Revert to the original pattern: E2E_TEST_UTILS_GIT_REF="${E2E_TEST_UTILS_GIT_REF:-}". Use the env var mechanism to test fork branches locally or in CI without committing the reference.

[rhdh-test-bot]

❌ Failed Nightly E2E Tests

Platform: ocp 4.20 | RHDH Version: 1.11 | Duration: 45m 2s
Passed: 208 | Failed: 17 | Flaky: 0 | Skipped: 66
Playwright Report | Build Log | Logs | Artifacts

[jrichter1]

/test e2e-ocp-helm-nightly

[rhdh-test-bot]

❌ Failed Nightly E2E Tests

Platform: ocp 4.20 | RHDH Version: 1.11 | Duration: 47m 26s
Passed: 204 | Failed: 17 | Flaky: 0 | Skipped: 71
Playwright Report | Build Log | Logs | Artifacts

[jrichter1]

/test e2e-ocp-helm-nightly

[rhdh-test-bot]

❌ Failed Nightly E2E Tests

Platform: ocp 4.20 | RHDH Version: 1.11 | Duration: 45m 25s
Passed: 212 | Failed: 9 | Flaky: 0 | Skipped: 71
Playwright Report | Build Log | Logs | Artifacts

[openshift-ci]

@jrichter1: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-ocp-helm-nightly	4cbaab8	link	false	/test e2e-ocp-helm-nightly

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

[fullsend-ai-retro]

🤖 Finished Retro · ✅ Success · Started 1:44 PM UTC · Completed 1:55 PM UTC

Commit: 4cbaab8 · View workflow run →

[fullsend-ai-retro]

Retro: PR #3355 — test: new auth providers for e2e

Verdict: Workflow operated correctly. No novel proposals warranted.

Timeline

PR #3355 was a short-lived human-authored WIP PR by jrichter1, opened and closed on 2026-08-21 without merging. It hardcoded E2E_TEST_UTILS_GIT_REF in run-e2e.sh to point at a personal fork (jrichter1/rhdh-e2e-test-utils#auth-providers) to test auth provider changes for E2E utilities. The PR was a deliberate test vehicle for work in the companion rhdh-e2e-test-utils PR #150.

What went well

• Review agent correctly identified the hardcoded personal fork reference as a high-severity correctness issue, requested changes, and applied the do-not-merge/work-in-progress label — all within ~2 minutes at $0.51 cost. The review was accurate, actionable, and appropriately calibrated.
• Routing worked correctly: the opened event dispatched to review, the agent's own pull_request_review event was correctly no-oped, and the closed event dispatched to retro.
• Label application was appropriate: the agent deduced WIP intent from the PR description and diff content even though the PR was not a draft and had no labels at open time.

Evidence for existing issues

• Skip retro for unmerged human PRs with only review-agent interaction fullsend-ai/fullsend#2471 (Skip retro for unmerged human PRs with only review-agent interaction): This retro run is itself evidence for that issue. PR test: new auth providers for e2e #3355 was a human-authored WIP PR, closed without merge, where the only agent involvement was a single routine review with no rework. The retro adds no value here — the review was correct and efficient, there were no iterations, and the PR had no production impact.
• Skip review dispatch for PRs with DO NOT MERGE / WIP title signals fullsend-ai/fullsend#2390, chore(app-defaults): bump source ref for electron-to-chromium pin (0.0.3) #3205 (Skip or short-circuit review for WIP/DNM PRs): The review ran because the PR had no labels or draft status at open time. The review still provided genuine value (flagging the fork reference and applying the WIP label), so skipping entirely may not be the right tradeoff — but the existing issues cover this design space.
• Skip review dispatch when PR is in draft state fullsend-ai/fullsend#1715 (Skip review for draft PRs): Not directly applicable since this PR was not a draft, but related — if the author had used draft status, the review could have been avoided.