Skip to content

🧪 ci: trigger v2 test guards on guarded files (#5388) - #5397

Merged
clubanderson merged 1 commit into
kubestellar:v4from
Danathar:fix/5388-property-assertions
Sep 1, 2026
Merged

🧪 ci: trigger v2 test guards on guarded files (#5388)#5397
clubanderson merged 1 commit into
kubestellar:v4from
Danathar:fix/5388-property-assertions

Conversation

@Danathar

Copy link
Copy Markdown
Contributor

Addresses #5388

Problem

The v2 Tests workflow is the PR gate that executes go test ./pkg/..., but its pull-request path filter only covered src/**, dashboard/openapi.json, and the workflow itself. A deliberate sweep of package tests found more guards whose inputs live outside src/:

  • bin/agent-launch.sh, bin/contributor-agent.sh, and bin/contributor-relay.sh are read by stderr-scrubbing, pane-working-directory, and relay-protocol tests;
  • bin/gh-app-token.sh and bin/hive-open-pr.sh are executed by credential-isolation and PR-request tests;
  • bin/hive-setup.sh and bin/hive-prereq-check.sh are inspected by the fresh-install branch guard;
  • config/backends.conf is sourced by the shell/Go backend and host-state deny-list parity tests;
  • Justfile is inspected by contributor-recipe behavior tests.

A PR changing only one of those shipped files did not start v2 Tests. The relevant assertion could therefore be correct, present, and permanently green while the guarded side regressed—the same path-exemption failure described in #5388 for dashboard/openapi.json.

Root cause

The workflow filter followed the location of the Go tests (src/**) rather than the complete set of files whose properties those tests assert. v2-ci.yml does run for bin/**, but it builds and vets without running the package unit suite, so that separate workflow did not close the gap. config/backends.conf and Justfile were outside the package-test trigger as well.

Implementation

  • .github/workflows/v2-tests.yml now includes bin/**, config/**, and Justfile in the PR path filter. The existing src/**, OpenAPI contract, and workflow-self entries remain intact.
  • src/pkg/github/ci_trigger_contract_test.go parses the real workflow and evaluates whether every current repository-external package-test input would cause v2 Tests to run.
  • The matcher checks the effective changed-file property rather than requiring a particular YAML shape. It accepts exact entries or broader equivalent globs, implements the recursive /** form used here, and honors GitHub's ordered negative-pattern behavior.

The directory filters are intentionally broader than today's exact filename list. Multiple packages already consume several bin/ files, and an exact allow-list would recreate the same maintenance trap when the next script-backed package test is added. The additional CI cost is limited to PRs that change shipped contributor scripts/configuration or the root Justfile—the inputs whose package guards need a chance to fail.

Regression evidence

TestV2TestsTriggersForExternalPackageTestInputs covers eleven external inputs/properties, including the workflow itself. TestV2TestsPathMatcherReproducesThePre5388Gap carries the old filter as a failure-direction control and confirms it does not match representative bin/, config/, or Justfile changes; it also proves a later negative glob excludes an earlier positive match.

The failure direction was also demonstrated against the real workflow during development. Temporarily removing the three new trigger entries made the contract exit 1 with nine named failures, including:

a PR changing only bin/contributor-relay.sh does not trigger v2-tests; the relay protocol compatibility guard cannot fail
a PR changing only config/backends.conf does not trigger v2-tests; the shell and Go backend parity guard cannot fail
a PR changing only Justfile does not trigger v2-tests; the contributor recipe behavior guard cannot fail

Restoring the entries returned the same test to green, and the committed tree is clean.

Verification

  • go test ./pkg/github -run '^TestV2Tests' -count=1 — PASS
  • targeted existing guards across ./pkg/agent ./pkg/config ./pkg/dashboard ./pkg/github with -short -count=1 — PASS
  • go test ./pkg/github -short -count=1 — PASS (full package, 44.758s)
  • go vet ./pkg/github — PASS
  • bash src/scripts/check-release-lines.sh — PASS; every pinned workflow still covers v2,v4
  • git diff --check — PASS

The first sandboxed cross-package attempt could not download missing declared Go modules, and the first full-package attempt could not bind the loopback sockets used by httptest; rerunning with the required network/loopback access passed as recorded above. These were environment restrictions, not test failures in the change.

Scope

This PR closes the concrete path-filter exemptions found in the requested sweep and prevents their current form from returning. It does not change runtime behavior, make unrelated conditional test skips fatal, or claim that every shape-oriented assertion in the repository has been converted in one change. No changelog entry is included because this is CI and regression-test wiring, not a user-visible product change.

— hive: backend=codex

Package tests inspect shipped files outside src/, but v2-tests only ran for src changes (plus the recently added OpenAPI exception). A PR changing Justfile, config/backends.conf, or the bin scripts could therefore bypass the very parity and behavioral assertions intended to police it.

Extend the pull-request filter to those guarded roots and add a trigger-contract test over every current external package-test input. The test evaluates the effective changed-file property, including ordered negative patterns, and carries the pre-fix filter as a failure-direction control so an always-true matcher cannot make the guard vacuously green.

Signed-off-by: Danathar <6772335+Danathar@users.noreply.github.com>
@Danathar
Danathar requested a review from clubanderson as a code owner August 31, 2026 23:26
@kubestellar-prow kubestellar-prow Bot added the dco-signoff: yes Indicates the PR's author has signed the DCO. label Aug 31, 2026
@kubestellar-prow

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign hanthor for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubestellar-prow kubestellar-prow Bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Aug 31, 2026
@kubestellar-hive

Copy link
Copy Markdown
Contributor

Quality verification (live hive host, go1.x): fetched pull/5397/head and ran go test -count=1 ./pkg/github/ -run TestV2Tests — all 11 trigger-contract subtests plus the pre-#5388 negative control pass. The matcher correctly implements ordered include/exclude and GitHub's recursive /** semantics, and the failure-direction control proves the contract can go red. No concerns from the quality side.

Filed by quality agent (hold-gated mode).

🐝 Hive Agent: quality | Instance: hosted-available-oke-11-placeholder-r05x | SHA: unknown

— hive: agent=quality backend=copilot model=claude-fable-5

@clubanderson
clubanderson merged commit e4bef31 into kubestellar:v4 Sep 1, 2026
42 of 43 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dco-signoff: yes Indicates the PR's author has signed the DCO. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants