🧪 ci: trigger v2 test guards on guarded files (#5388) - #5397
Conversation
Package tests inspect shipped files outside src/, but v2-tests only ran for src changes (plus the recently added OpenAPI exception). A PR changing Justfile, config/backends.conf, or the bin scripts could therefore bypass the very parity and behavioral assertions intended to police it. Extend the pull-request filter to those guarded roots and add a trigger-contract test over every current external package-test input. The test evaluates the effective changed-file property, including ordered negative patterns, and carries the pre-fix filter as a failure-direction control so an always-true matcher cannot make the guard vacuously green. Signed-off-by: Danathar <6772335+Danathar@users.noreply.github.com>
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
Quality verification (live hive host, go1.x): fetched Filed by quality agent (hold-gated mode).🐝 Hive Agent: — hive: agent=quality backend=copilot model=claude-fable-5 |
Addresses #5388
Problem
The
v2 Testsworkflow is the PR gate that executesgo test ./pkg/..., but its pull-request path filter only coveredsrc/**,dashboard/openapi.json, and the workflow itself. A deliberate sweep of package tests found more guards whose inputs live outsidesrc/:bin/agent-launch.sh,bin/contributor-agent.sh, andbin/contributor-relay.share read by stderr-scrubbing, pane-working-directory, and relay-protocol tests;bin/gh-app-token.shandbin/hive-open-pr.share executed by credential-isolation and PR-request tests;bin/hive-setup.shandbin/hive-prereq-check.share inspected by the fresh-install branch guard;config/backends.confis sourced by the shell/Go backend and host-state deny-list parity tests;Justfileis inspected by contributor-recipe behavior tests.A PR changing only one of those shipped files did not start
v2 Tests. The relevant assertion could therefore be correct, present, and permanently green while the guarded side regressed—the same path-exemption failure described in #5388 fordashboard/openapi.json.Root cause
The workflow filter followed the location of the Go tests (
src/**) rather than the complete set of files whose properties those tests assert.v2-ci.ymldoes run forbin/**, but it builds and vets without running the package unit suite, so that separate workflow did not close the gap.config/backends.confandJustfilewere outside the package-test trigger as well.Implementation
.github/workflows/v2-tests.ymlnow includesbin/**,config/**, andJustfilein the PR path filter. The existingsrc/**, OpenAPI contract, and workflow-self entries remain intact.src/pkg/github/ci_trigger_contract_test.goparses the real workflow and evaluates whether every current repository-external package-test input would causev2 Teststo run./**form used here, and honors GitHub's ordered negative-pattern behavior.The directory filters are intentionally broader than today's exact filename list. Multiple packages already consume several
bin/files, and an exact allow-list would recreate the same maintenance trap when the next script-backed package test is added. The additional CI cost is limited to PRs that change shipped contributor scripts/configuration or the rootJustfile—the inputs whose package guards need a chance to fail.Regression evidence
TestV2TestsTriggersForExternalPackageTestInputscovers eleven external inputs/properties, including the workflow itself.TestV2TestsPathMatcherReproducesThePre5388Gapcarries the old filter as a failure-direction control and confirms it does not match representativebin/,config/, orJustfilechanges; it also proves a later negative glob excludes an earlier positive match.The failure direction was also demonstrated against the real workflow during development. Temporarily removing the three new trigger entries made the contract exit 1 with nine named failures, including:
Restoring the entries returned the same test to green, and the committed tree is clean.
Verification
go test ./pkg/github -run '^TestV2Tests' -count=1— PASS./pkg/agent ./pkg/config ./pkg/dashboard ./pkg/githubwith-short -count=1— PASSgo test ./pkg/github -short -count=1— PASS (full package, 44.758s)go vet ./pkg/github— PASSbash src/scripts/check-release-lines.sh— PASS; every pinned workflow still coversv2,v4git diff --check— PASSThe first sandboxed cross-package attempt could not download missing declared Go modules, and the first full-package attempt could not bind the loopback sockets used by
httptest; rerunning with the required network/loopback access passed as recorded above. These were environment restrictions, not test failures in the change.Scope
This PR closes the concrete path-filter exemptions found in the requested sweep and prevents their current form from returning. It does not change runtime behavior, make unrelated conditional test skips fatal, or claim that every shape-oriented assertion in the repository has been converted in one change. No changelog entry is included because this is CI and regression-test wiring, not a user-visible product change.
— hive: backend=codex