You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
As an OpenShell maintainer using gator to validate a Helm or Kubernetes PR, I want the selected test labels and bot instructions to enable the intended CI jobs, so that I can obtain runtime coverage without reverse-engineering workflow conditions.
Problem Statement
Gator's test-label guidance, the E2E Label Help bot, and CI.md disagree with the executable Branch E2E Checks workflow.
Observed on PR #3459 (feat(helm): make cluster-scoped RBAC optional) at head e876b74eb9a83ecda9d1b1e349981cfea84c6cef:
Gator applied test:e2e-kubernetes without test:e2e. Its instructions explicitly recommend the Kubernetes-specific label for Helm, namespace, controller, and related changes.
The help bot told the maintainer to rerun all jobs and promised Kubernetes HA and credential-driver E2E.
Attempt 2 of the linked workflow completed successfully after metadata resolution, with the build and E2E jobs skipped. Its metadata log contained LABELS_JSON: ["test:e2e-kubernetes","gator:blocked"].
The workflow enables standard kubernetes-e2e through test:e2e. It hardcodes both run_kubernetes_ha_e2e and run_kubernetes_credential_drivers_e2e to false; test:e2e-kubernetes therefore enables no suite in this revision.
There is no label-promotion step that adds test:e2e. The trusted pull-request/3459 mirror already matched the PR head, so another /ok to test <SHA> comment would not resolve this selection mismatch. Gator remained blocked awaiting test dispatch; this report does not claim it declared the PR validated.
Impact / Why This Matters
Maintainers follow actionable bot instructions and spend another CI attempt without obtaining the advertised coverage. A successful workflow conclusion can obscure that every runtime test was skipped, and gator's handoff remains blocked.
The immediate workaround for standard Kubernetes coverage is to add test:e2e and rerun all jobs. That enables the broader core E2E suite; it does not enable the disabled HA or credential-driver jobs. The workaround requires inspecting workflow source and knowing which similarly named label actually controls the desired tests. It does not correct guidance for subsequent PRs or deliver the specialized coverage the bot promises.
Acceptance Criteria
Gator's label-selection guidance for Helm/Kubernetes changes agrees with the suites actually available in the applicable workflow revision; requesting standard Kubernetes E2E selects the label that enables it.
E2E Label Help and CI.md accurately describe each supported label. If HA or credential-driver suites remain disabled, they do not promise those jobs will run, and they explain any supported alternative.
Following the documented label-and-rerun sequence on a current-head PR makes the intended test jobs eligible to execute after successful prerequisite builds.
Gator distinguishes an intended suite that ran from one whose jobs were all skipped, even when the enclosing workflow concluded successfully, and reports an actionable reason for missing coverage.
Focused regression coverage detects drift between supported test labels, suite-selection conditions, and the help instructions, including the Kubernetes-label-only case.
Follow E2E Label Help's instruction to select Re-run all jobs on the existing Branch E2E Checks run.
Inspect metadata and job conclusions: the Kubernetes label is present, but run_core_e2e is false and the specialized Kubernetes switches are also false. Build and test jobs are skipped.
Compare the advertised HA/credential-driver coverage with those workflow conditions.
PR #3459 subsequently received test:e2e manually; use the preserved attempt-2 evidence rather than its current labels to inspect the original failure.
User Story
As an OpenShell maintainer using gator to validate a Helm or Kubernetes PR, I want the selected test labels and bot instructions to enable the intended CI jobs, so that I can obtain runtime coverage without reverse-engineering workflow conditions.
Problem Statement
Gator's test-label guidance, the E2E Label Help bot, and
CI.mddisagree with the executable Branch E2E Checks workflow.Observed on PR #3459 (
feat(helm): make cluster-scoped RBAC optional) at heade876b74eb9a83ecda9d1b1e349981cfea84c6cef:test:e2e-kuberneteswithouttest:e2e. Its instructions explicitly recommend the Kubernetes-specific label for Helm, namespace, controller, and related changes.LABELS_JSON: ["test:e2e-kubernetes","gator:blocked"].kubernetes-e2ethroughtest:e2e. It hardcodes bothrun_kubernetes_ha_e2eandrun_kubernetes_credential_drivers_e2etofalse;test:e2e-kubernetestherefore enables no suite in this revision.There is no label-promotion step that adds
test:e2e. The trustedpull-request/3459mirror already matched the PR head, so another/ok to test <SHA>comment would not resolve this selection mismatch. Gator remained blocked awaiting test dispatch; this report does not claim it declared the PR validated.Impact / Why This Matters
Maintainers follow actionable bot instructions and spend another CI attempt without obtaining the advertised coverage. A successful workflow conclusion can obscure that every runtime test was skipped, and gator's handoff remains blocked.
The immediate workaround for standard Kubernetes coverage is to add
test:e2eand rerun all jobs. That enables the broader core E2E suite; it does not enable the disabled HA or credential-driver jobs. The workaround requires inspecting workflow source and knowing which similarly named label actually controls the desired tests. It does not correct guidance for subsequent PRs or deliver the specialized coverage the bot promises.Acceptance Criteria
CI.mdaccurately describe each supported label. If HA or credential-driver suites remain disabled, they do not promise those jobs will run, and they explain any supported alternative.Reproduction Steps
test:e2e-kubernetesas the test label.run_core_e2eis false and the specialized Kubernetes switches are also false. Build and test jobs are skipped.PR #3459 subsequently received
test:e2emanually; use the preserved attempt-2 evidence rather than its current labels to inspect the original failure.Environment
cb93f62bfe7aef5cae92c8e46cba4ac9b00f9899.0.0.117-dev.188+gcb93f62bf, Docker-backed gateway on macOS. The defect is in repository CI/agent instructions, not the local gateway.pull-request/3459, run35344440601, attempt 2, heade876b74eb9a83ecda9d1b1e349981cfea84c6cef.Evidence