ci: breaking-change gate on the pg-pkg OpenAPI contract (oasdiff) - #269
Conversation
The gate itself is an oasdiff job in .github/workflows/api-diff.yml, which the App cannot push, so the YAML ships as a comment on this PR for a maintainer to apply. What is committed here is the docs half: COMPATIBILITY.md and CLAUDE.md now say the /v2 rules are still a review rule, name the job and where its YAML is waiting, and record how to reproduce the verdict locally (oasdiff v1.26.1, the version the pinned action runs). Refs #249. Part of #247 (workstream C).
2ac9ac4 to
1593102
Compare
|
There was a problem hiding this comment.
VERDICT: request-changes
Rules Dobby 2, cycle 1. Rule sweep over 14 applicable rules plus Review Dobby 2's findings, all re-verified against HEAD 1593102.
One blocking issue: the gate does not catch three of the changes its own comment and COMPATIBILITY.md promise it catches. I reproduced this independently — built oasdiff v1.26.1 (the engine v0.1.10 ships) and mutated the real pg-pkg/api-description.yaml, using the step's exact flags:
| mutation | --fail-on ERR, no include-checks |
with include-checks |
|---|---|---|
401 → 403 on GET /v2/request/key/{timestamp} |
exit 0, no finding | response-non-success-status-removed ERR, exit 1 |
TIMEOUT dropped from SessionStatus |
exit 0, no finding | response-property-enum-value-removed, 3 ERRs, exit 1 |
optional key removed from KeyResponse |
exit 0 (response-optional-property-removed is WARN) |
exit 1 only at --fail-on WARN |
The workflow header comment promises "a changed status code for a condition a client already handles" and "a route or field removed or renamed". Neither holds today. The first two are one input away — add to the oasdiff step:
fail-on: ERR
include-checks: response-non-success-status-removed,response-property-enum-value-removed
review: falseThe third needs a decision rather than a flag, and it is the one worth a maintainer's attention: in this spec only status is ever required, so key, proofStatus, pubSignKey, privSignKey and the rest are all optional, and removing any of them is WARN. key is the IBE user secret key — the payload the endpoint exists to return. Two honest options:
- Raise to
fail-on: WARN. That is 31 more checks, and most are thingsCOMPATIBILITY.mdalready forbids (request-parameter-removed,response-body-media-type-schema-removed). Some are noisy. - Keep
ERRand narrow the promise to "a removed route, or a removed/renamed required response field" in both the workflow comment andCOMPATIBILITY.md.
Either way the two places that currently overpromise should match what the gate actually does.
Also worth fixing in the same push (non-blocking)
- Two bare cross-repo refs in the PR body (
cross-repo-link-format). Line 27 reads "That happened on cryptify#201 itself: #200 merged" —cryptify#201does not auto-link across repos, and the bare#200resolves to postguard#200, a real unrelated PR ("feat(pkg): proxy IRMA /statusevents SSE endpoint"), so it links a reader to the wrong thing. Useencryption4all/cryptify#201andencryption4all/cryptify#200. - The alias gap deserves one clause in
CLAUDE.md. The gate only sees paths the document lists, and the spec deliberately documents canonical paths only (its own "Path prefix aliases" note, lines 16-20 — I confirmed no/v2/irmapath appears anywhere in the spec). So dropping the/v2/irma/...handlers, the aliasCOMPATIBILITY.mdpromises keeps working until #257's deprecation runs, passes the gate silently. The enforcement paragraph sits directly under the alias paragraph, which reads as though it covers it. - The pending wording is correct now and goes stale on apply.
patch-in-comment-gate-is-not-live-yetrequires the tightening to happen in this same PR once a maintainer commits the YAML onto the branch — the PR body already lists it as a pre-merge TODO, which is the right call. Flagging it so it does not merge with the pending tense baked in.
What the sweep cleared
review: false set and the action sha-pinned with the tag in a trailing comment (oasdiff-action-uploads-specs-by-default — the sha pin is stronger than a tag pin and matches dorny/paths-filter in build.yml); no on: paths: filter, so the job always reports and can be a required check (path-filtered-job-cannot-be-a-required-check); edited present in the trigger, which the base-sha-dependent verdict needs (pull-request-default-types-exclude-edited); no run: blocks, so no pipefail trap; allow-external-refs at its SSRF-guarding default; reviewer assigned; conventional-commit title green; repo knowledge in the repo's CLAUDE.md rather than shared rules. Refs #249 rather than Closes #249 is right here — merging this does not make the gate live.
Tests: not run, and correctly so — the committed diff is two markdown files. Repo CI is 23 pass / 0 fail.
The gate as reviewed promised more than fail-on: ERR delivers. Measured against oasdiff v1.26.1 on the real spec, ERR silently passed six changes COMPATIBILITY.md forbids: a changed status code, a dropped response enum value, a removed and a renamed optional response property, a removed non-success status, and a removed request parameter. Two of those (response-non-success-status-removed, response-property-enum-value-removed) rate ERR but are opt-in, so they only run when named in include-checks. The rest are WARN, and on this spec that is most of the "no removing a route or a field" rule: only `status` is ever `required`, so `key`, `proofStatus`, `pubSignKey` and the rest are optional. So the gate now runs fail-on: WARN plus those two checks. Of the 31 WARN checks, response-property-enum-value-added is the only one that fires on something COMPATIBILITY.md did not already forbid, so that document now names a new response enum value as non-additive too. pg-pkg/tests/api_gate.rs is the regression test: it mutates the spec 18 ways, runs the engine with the flags the action's entrypoint builds, and asserts which mutations the gate stops. It fails on the old settings with all six gaps named, and skips when oasdiff is not on PATH, which is the case in CI. Refs #249, #242. Part of #247 (workstream C). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The test's module docs described the gate in the present tense, which is the same overpromise the reviewed docs had: the workflow YAML is still in a PR comment waiting for workflows: write. CLAUDE.md and COMPATIBILITY.md already say so; this file now does too. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
COMPATIBILITY.md gained "adding a value to a response enum" for this gate, and it was the one rule api_gate.rs did not cover. It holds only at WARN (measured on oasdiff v1.26.1: exit 0 at ERR+include-checks, exit 1 at WARN+include-checks), so a revert to fail-on: ERR dropped it with the test still green -- exactly the drift the test exists to catch. At ERR the test now reports four wrong verdicts instead of three, the fourth being this one.
There was a problem hiding this comment.
VERDICT: approve
Rules Dobby 2, cycle 2. Reviewed HEAD ffc5bd4, then pushed f0edf44 with the two nit fixes below. No blocking findings, so this is signed off and flipped out of draft.
Cycle 1's blocking finding is closed
The gate now delivers what its comment and COMPATIBILITY.md promise. I rebuilt the measurement rather than reading it off the PR body: installed oasdiff v1.26.1 locally and replayed all ten breaking mutations from api_gate.rs with the exact flags the action's entrypoint builds. At fail-on: ERR plus include-checks, three pass that should not (a removed optional response property, a renamed one, a removed request parameter). At WARN plus include-checks all ten fail. Running cargo test --test api_gate with the real engine on PATH: both tests pass, all 19 mutations plus the unmutated baseline.
Rules checked
14 rules apply to this diff. All compliant.
oasdiff-action-uploads-specs-by-default is the one with teeth here, and the YAML satisfies every part of it: review: false, action sha-pinned with the tag in a trailing comment, allow-external-refs left at its default, both opt-in checks named, and the tier pinned in a committed test rather than only in the workflow.
patch-in-comment-gate-is-not-live-yet: CLAUDE.md, COMPATIBILITY.md and the test header all say the gate is not running yet and name the comment it waits in. The PR body also commits to tightening those lines in this PR once a maintainer applies the YAML, which is what the rule asks for.
path-filtered-job-cannot-be-a-required-check: no on: paths: filter, and the comment says why.
pull-request-default-types-exclude-edited: edited is in the trigger, and the job does read pull_request.base.sha.
actions-default-shell-has-no-pipefail: no run: blocks, so there is no shell to guard. actionlint 1.7.7 is clean on the YAML after my edit to it.
ci-gate-can-ride-an-existing-build-script: the test rides Test workspace (pkg), which the matrix runs with --all-features. Confirmed against build.yml rather than taken from the PR body.
rust-run-cargo-fmt-before-push: cargo fmt and cargo clippy --all-targets --all-features -- -D warnings both clean on pg-pkg, before and after my commit.
writing-rules: zero banned words, zero em dashes in the added prose, three bold spans across 2945 added words.
Also checked and clean: conventional-commit-pr-titles (the repo runs the job and it passes), cross-repo-link-format, single-repo-lessons-belong-in-that-repos-claude-md, grep-a-construction-site-before-user-visible-claim, tests-required-on-fixes, promised-vs-delivered. No advisory or embargo is touched, so the ready flip is safe.
Two nits, both fixed on the branch
Neither blocks, so instead of sending this back I fixed them in f0edf44 and made the surrounding claims match.
The first is a missing mutation for the newest rule. Nothing covered adding a response enum value, which is the one rule COMPATIBILITY.md gains in this PR and the only one WARN alone enforces. Measured on the real spec: adding EXPIRED to SessionStatus exits 0 at ERR+include-checks and exits 1 at WARN+include-checks. So the revert path the PR body offers dropped the newest rule while the test stayed green, which is the drift the test exists to prevent. api_gate.rs now pins it as a 19th mutation.
The second is the revert sentence's count. It said four Gate::Stops expectations stop holding at ERR; before this commit it was three, and the workflow comment and CLAUDE.md both said three. Adding the enum case makes four correct, so I kept the number and named which four. I also re-measured the quoted failure output for the old settings: it is now 7 of 20, not 6 of 19.
Adding the 19th case moved two numbers written before it existed, so those moved too: CLAUDE.md's "18 ways" to 19, and in the workflow comment the "three of the changes COMPATIBILITY.md forbids" paragraph to four, with the enum row added to its table. The gate inputs in that comment are byte-identical to what cycle 1 reviewed; only comment lines changed.
For the maintainer
Nothing above changes what you need to do. Paste the YAML from the comment onto this branch, then the two "not live yet" doc lines tighten to present tense before merge, and API breaking changes (oasdiff) can go into branch protection for #247.
Dobby's YAML from the #269 comment, applied verbatim (the App cannot push workflows). Landing it here rather than on main so the gate self-tests on this PR, and tightening the two doc lines that described it as not-yet-live.
|
Workflow applied onto this branch (so the gate self-tests here, as you suggested), and both doc lines tightened to present tense: COMPATIBILITY.md no longer says "written but not running yet", and CLAUDE.md no longer points at a spent PR comment. Your |
|
/dobby fix conflicts and review if the workflow is implemented as intended |
|
Got it — I'm on PR #269 now. I'll resolve the merge conflicts, then review the changes to check the workflow is implemented as intended and push fixes if needed. |
# Conflicts: # COMPATIBILITY.md
…nstants api_gate.rs's header named a hole that only existed while the YAML was uncommitted: FAIL_ON and INCLUDE_CHECKS are what the test runs oasdiff with, the workflow step is what CI runs it with, and nothing compared the two. The verdict test skips on every runner, so a `fail-on` edited down to ERR in the workflow alone lands green with the whole suite passing. The new test reads both values back out of the YAML and needs no engine, so it runs in CI. Also drops the rest of the header's not-committed-yet wording (the merge before this one did the same for COMPATIBILITY.md), and records in CLAUDE.md that the remote's workflow-path rejection is per commit diff, so merging a main that changed build.yml is pushable.
The merge in 94e6ed5 hit a conflict where main's two new semver bullets and this branch's two oasdiff bullets both landed under the Docker one, and took main's side whole, so the gate this PR adds ended up documented nowhere in CLAUDE.md. Put them back in place, unchanged, alongside the semver pair. Also notes on the workflow-push bullet that the App did push a merge carrying main's build.yml changes (ce0fc59 on this branch), so a sync is worth trying before treating it as maintainer-only.
|
Conflicts resolved and the workflow reviewed against the action's source. Branch is mergeable, all 26 checks pass, and the gate ran for the first time. The conflict, and why no check had ever reportedOne hunk: the link-reference block at the bottom of The conflict is why the gate had not run. GitHub cannot build What I verified about the stepAgainst
One gap, closed
94e6ed5 dropped the gate's CLAUDE.md entryThe "Update branch" merge hit a conflict where main's two While there, one correction to flag rather than bury, since it is your sentence: the bullet now says a merge that has to touch Not changedThe step uses |
…means Hit on this branch: the same docs-only sha passed all three browser jobs at 07:45 and failed at 07:48. The visible error, `missing field 'chunk'`, is the test runner choking on a truncated webdriver reply, so it reads like a harness version mismatch; the actual cause is the renderer timeout logged one line above it. Also notes that only chrome failed and the other two were cancelled by the fail-fast matrix but reported as failures, and that the App cannot re-run a workflow.
|
Two things after the last push, both now settled. The applied YAML is byte-identical to what was in the comment. Checked rather than assumed: extracted the fenced block from comment 5103413183 and diffed it against The three red wasm jobs were a flake, and the branch is green now. Same sha e86c71b ran CI twice, once from the push and once from my PR-body edit ( All 26 checks pass on 6aedefd. Nothing here needs a rebase. |
`the_workflow_step_matches_the_pinned_inputs` read `fail-on` and `include-checks` back out of the workflow but not the action ref, which is the third thing the pinned verdicts depend on: `breaking@0ab8ad2` is v0.1.10, whose Dockerfile is `FROM tufin/oasdiff:v1.26.1`, and every verdict in this file was measured against v1.26.1. So bumping the action left all three tests green while the verdicts quietly stopped describing CI -- the same fail-open the `fail-on` pin exists to prevent, and the file's header claims its verdict is CI's verdict. Assert the workflow contains the ref too. Verified by bumping the ref in the workflow: the test fails and names it.
pg-pkg/tests/api_gate.rs locates blocks in the spec with multi-line raw-string anchors. rustc normalises CRLF to LF inside raw strings, but fs::read_to_string of the spec keeps whatever git wrote, so on a Windows checkout (core.autocrlf=true is the Git for Windows default) every multi-line anchor misses. Reproduced rather than argued: converting the spec to CRLF fails every_mutation_still_applies with "anchor is not unique in the spec", i.e. the one gate test that runs without the external engine, so a Windows contributor gets a red suite on a clean tree with a message blaming the spec. CI is ubuntu/macos only, so it never catches this. Mark the spec -text, which keeps stored bytes equal to checked-out bytes in both directions.
There was a problem hiding this comment.
VERDICT: approve
Rules Dobby 2, cycle 3. Reviewed HEAD 6aedefd, then pushed 3ff6d8e with two nit fixes. No blocking findings. All 26 checks pass on the new sha, and the PR is already out of draft.
Is the workflow implemented as intended?
Yes, with two deviations from #249, and until this cycle the PR body only admitted one.
The gate itself does what its comment and COMPATIBILITY.md say it does. Cycle 1's blocking finding (the prose promised more than fail-on: ERR delivered) was closed in cycle 2 by re-measuring against the real engine at WARN plus the two opt-in checks. Nothing since has touched api-diff.yml or the pinned verdicts: the delta from cycle 2 is 8f26cf7 (@rubenhensen applying the YAML), 44d6379 (the inputs test), two base merges, and two docs commits. The gate is green and the job reports on every PR, so it can go into branch protection for #247.
The second deviation is the one worth a decision from you. #249 asks for "a new versioned route with the old one left running, or an explicit override label". Only the first is implemented. Dropping a label bypass on a required check is defensible, and /v3 is a real escape hatch here because pg-pkg's routes are versioned, but it is a narrowing of the brief that the body did not mention. It does now.
The conflict resolution
Clean, and I checked it by patch-content identity rather than by reading the hunks. The branch's contribution before the merge and after it are the same +/- lines with one exception: 94e6ed5 took main's side of CLAUDE.md whole and so dropped all three of this branch's new bullets. e86c71b restored them, and both are present at HEAD. Worth recording because that is the failure mode of that merge, and nothing in the conflict hunks would have shown it.
git diff against the base is 769 lines, so this is not a no-op PR. ce0fc59 is also the measured exception to the workflows: write block that the CLAUDE.md bullet now documents: a merge that takes the base's build.yml side whole is pushable by the App, because the blob it commits already exists in the repo.
Rules checked
15 rules apply. All compliant after the two fixes below.
oasdiff-action-uploads-specs-by-default is the one with teeth, and the step satisfies it: review: false, action sha-pinned with the tag in a trailing comment, allow-external-refs at its default, both opt-in checks named, fail-on: WARN, and the tier pinned in a test that reads the workflow file rather than in a bare constant.
patch-in-comment-gate-is-not-live-yet now cuts the other way: the YAML is applied, so present-tense docs are correct. COMPATIBILITY.md and the api_gate.rs header were tightened accordingly, which is what the rule asks for in the same PR.
path-filtered-job-cannot-be-a-required-check: no on: paths: filter, and the comment says why. pull-request-default-types-exclude-edited: edited is in the trigger, and the job does read pull_request.base.sha. actions-default-shell-has-no-pipefail: no run: blocks, so no shell to guard.
Also checked and clean: conflict-resolution-check-noop, verify-a-merge-by-patch-content-identity, workflow-file-conflict-is-unpushable-and-kills-ci, single-repo-lessons-belong-in-that-repos-claude-md, rust-run-cargo-fmt-before-push, ci-gate-can-ride-an-existing-build-script, tests-required-on-fixes, no-justification-paragraphs-for-simple-changes, writing-rules, never-self-merge-prs. No advisory or embargo is touched.
Two nits fixed on the branch
Neither blocks, so I fixed them rather than sending the PR back.
The action ref was not pinned (4334f9a). the_workflow_step_matches_the_pinned_inputs read fail-on and include-checks back out of the workflow but not the action ref, which is the third thing the verdicts depend on: breaking@0ab8ad2 is v0.1.10, whose Dockerfile is FROM tufin/oasdiff:v1.26.1, and every verdict in the file was measured against v1.26.1. So bumping the action left all three tests green while the verdicts quietly stopped describing CI, which is the fail-open the fail-on pin exists to prevent. Verified by bumping the ref in the workflow: the test now fails and names it.
The spec's line endings were not pinned (3ff6d8e). The mutation anchors are multi-line raw strings matched against the spec read off disk. rustc normalises CRLF to LF inside raw strings, but fs::read_to_string does not, so on a Windows checkout (core.autocrlf=true is the Git for Windows default) every multi-line anchor misses. Reproduced rather than argued: converting the spec to CRLF fails every_mutation_still_applies with anchor is not unique in the spec, which is the one gate test that runs without the external engine, so a Windows contributor gets a red suite on a clean tree with a message blaming the spec. CI is ubuntu/macos only and never sees it. Root .gitattributes now marks the spec -text. This is the same defect found on the sibling gate in encryption4all/cryptify#203.
cargo fmt --all -- --check clean, cargo clippy --all-targets --all-features -- -D warnings clean on pg-pkg, and the full pg-pkg suite passes (59 + 3 + 2).
Two nits left, both in the workflow file
Inline below. Both are one-line YAML edits that need your hands, since the App has no workflows: write on this repo. Neither changes behaviour, and neither blocks the merge.
Loop limit
This is cycle 3, so the PR is not going back to the coder regardless. Everything I could fix from a pushable file is fixed and verified; the remainder is the two inline YAML lines and the label question above, all of which are yours to decide.
| # Do not upload the two specs to oasdiff.com for a side-by-side | ||
| # review page. The default is `true`; the detection and the inline | ||
| # annotations work without it, so nothing leaves CI. | ||
| review: false |
There was a problem hiding this comment.
Nit, needs your hands (the App has no workflows: write): add github-token: '' to this with: block.
The input is left at its action default, which is ${{ github.token }}, so the job's GITHUB_TOKEN is handed to the third-party oasdiff container even though nothing uses it. Confirmed in the real run on this branch: the docker invocation carries -e "INPUT_GITHUB-TOKEN" and "***" as its last positional argument. Both post_review_comment call sites in the action's entrypoint at the pinned sha are guarded by [ "$review" != "false" ], so with review: false the token is never read.
That makes it a free removal, and it is the same threat model this step already reasons about two lines up, where persist-credentials: false is set on both checkouts because the workspace is mounted into that container. No behaviour change.
| # new prefix (/v3/...) with /v2 left running, and /v2 is retired later through | ||
| # the deprecation process in COMPATIBILITY.md, once postguard-ops#64 telemetry | ||
| # shows nobody calls it. A /v3 route added next to /v2 reads as additive, so | ||
| # this gate passes it. Reach for that before reaching for err-ignore. |
There was a problem hiding this comment.
Nit, and a question rather than a defect.
This line presents /v3 as the escape hatch while pointing at err-ignore as the thing not to reach for. err-ignore is a file of regexes; what #249 actually offered as the alternative to a versioned route was "an explicit override label", and this PR does not implement one.
Dropping the label is defensible: on a required check it makes the gate advisory for anyone who can apply a label, and because pg-pkg's routes are versioned, /v3 is a real alternative rather than a formality. But it is a second deviation from #249, and the PR body claimed only one until this review. I have added it to the body; if you agree with the choice, a clause saying so belongs here too, since this comment is where the next person hitting a red gate will look.
If you would rather have the label, it is if: "!contains(github.event.pull_request.labels.*.name, 'api-break-approved')" on the job, and it needs a maintainer commit for the same permissions reason.
|
Done — conflicts were already cleanly resolved on the branch (verified by patch-content identity, which caught that Implemented as intended? Mostly. The gate does what its comment and Along the way I fixed and pushed two nits: pinned the previously-floating oasdiff action ref in the gate test (it silently affected the verdicts), and added a root Posted the review as approve with two small inline YAML nits I can't push myself (no |
Adds the oasdiff breaking-change gate for
pg-pkg/api-description.yamlfrom #249, mirroring the one that landed in encryption4all/cryptify#201..github/workflows/api-diff.ymlis in the diff because @rubenhensen applied it in 8f26cf7; the App has noworkflows: writehere, so the YAML started life in a comment below. The rest of the diff ispg-pkg/tests/api_gate.rs, which pins what the gate's settings actually deliver, and theCOMPATIBILITY.mdrule the WARN setting adds.Two deviations from the spec on #249
fail-onisWARN, notERR. The request pinnedfail-on: ERR. I measured what that gives us and it is not what the issue asks for ("removed route/field, type change, new required field"), so the gate ships atWARN. Reverting is a one-line change; the table below is the measurement.Only one of the two escape hatches is implemented. #249 asks for "a new versioned route with the old one left running, or an explicit override label". This ships the first and drops the second. A label that bypasses a required check is a footgun: it makes the gate advisory for anyone who can apply a label, and the whole point of #249 is that the gate becomes a required check for #247. Because pg-pkg's routes are versioned,
/v3is a real escape hatch rather than a formality, which is what makes dropping the label affordable here (cryptify, with unversioned routes, has no such option). Worth a maintainer's explicit yes or no, since it is a narrowing of the brief and not just a setting. If you want the label anyway, it isif: "!contains(github.event.pull_request.labels.*.name, 'api-break-approved')"on the job, and it needs your hands: the App has noworkflows: write.Measured against oasdiff v1.26.1 on the real spec, with the flags the action's entrypoint builds.
ERR+incisfail-on: ERRwith the two opt-in checks named:401becomes403401response removedkey)Three things came out of this:
Two checks are opt-in.
response-non-success-status-removedandresponse-property-enum-value-removedrate ERR but only run when named, soinclude-checksis not optional for us.The rest are WARN, and on this spec that is most of the rule.
COMPATIBILITY.mdforbids "removing a route or a field". Onlystatusis everrequiredin this spec, sokey(the IBE user secret key/v2/request/keyexists to return),proofStatus,pubSignKeyand the rest are optional, and removing or renaming an optional response property is WARN. So is removing a request parameter. At ERR the response side of that rule is close to unguarded.WARN is nearly a restatement of
COMPATIBILITY.md. It adds 31 checks on top of ERR's 213. Of the 31, exactly one fires on something the document did not already forbid:response-property-enum-value-added. I ran nine additive changes throughWARNand that was the only one flagged; the other eight (new endpoint, new optional response property, new optional request property, new required response property, new optional query parameter, new response status, an edited description, and a/v3route beside/v2) all pass. Rather than suppress it,COMPATIBILITY.mdnow names a new response enum value as non-additive, because it is: a client that switches onSessionStatuswithout a default branch breaks on a value it has never seen, which is a/v3change.If you would rather keep the brief exactly, set
fail-on: ERRand flip the fourGate::Stopsexpectations inapi_gate.rsthat stop holding. The test will tell you which: the three optional-field removals in the table above, plus a new response enum value, which is the ruleCOMPATIBILITY.mdgains in this PR and the one that only WARN enforces.The regression test
pg-pkg/tests/api_gate.rsmutates the spec 19 ways and asserts, for each, whether the gate stops it. It runs the engine with the flags the action's entrypoint builds (--allow-external-refs=false --include-checks … --composed=false --fail-on …), so its verdict is CI's verdict.On the old settings it fails and names every gap:
On the new ones it passes. It is wired into a job that already exists rather than a new one: CI runs
cargo test --manifest-path pg-pkg/Cargo.toml --all-features, which picks it up. The verdict test needs theoasdiffbinary, which no runner has, so it skips there with a message saying how to install it. Two tests run regardless:every_mutation_still_applies, so a spec edit that strands one of the anchors still fails in CI, andthe_workflow_step_matches_the_pinned_inputs, which readsfail-onandinclude-checksback out of the workflow and fails when they no longer match the constants the verdicts were measured against. Locally, withgo install github.com/oasdiff/oasdiff@v1.26.1, all three do the real work.Unchanged from cryptify's copy
review: falseso nothing is uploaded to oasdiff.com, no path filter so the job always reports a status and can become a required check, base pinned topull_request.base.sha.Three deltas from cryptify's copy
types: [opened, synchronize, reopened, edited]. A base retarget firesedited, which is not in the default set, so without it a verdict computed against the old base stays attached to the unchanged head sha and still reports green. That happened on ci: add the oasdiff breaking-change gate cryptify#201 itself: docs: true up api-description.yaml with the mounted routes cryptify#200 merged, GitHub retargeted, no run fired, and the green came from an unrelated force-push twenty seconds later.build.ymlalready carries this trigger for the same reason.0ab8ad2with the tag in a trailing comment), likedorny/paths-filterinbuild.yml. This step is the verdict.persist-credentials: falseon both checkouts. The oasdiff action is a third-party container with the workspace mounted, and the spec is all it reads.cryptify's gate has the same two gaps, for the same reason: it was written before any of this was measured, and it has neither
include-checksnor a narrowed promise. Filed as encryption4all/cryptify#202.Notes for the maintainer
refs/pull/N/merge, so nopull_requestworkflow fired at all while the conflict stood.allow-external-refsstays at itsfalsedefault. Every$refin the spec is internal, checked.pull-requests: write: withreview: falsethe action never posts a comment.workflows: write), neither of which changes behaviour:github-token: ''to the oasdiff step'swith:block. The input defaults to${{ github.token }}, so the job's token is handed to the third-party container as a positional argument for no reason. Confirmed in the run on this branch: the docker invocation carries-e "INPUT_GITHUB-TOKEN"and"***"as its last argument. Bothpost_review_commentcall sites in the action's entrypoint at the pinned sha are guarded by[ "$review" != "false" ], so withreview: falsenothing reads it. It is the same reasoning as thepersist-credentials: falsetwo lines up, which is there because the workspace is mounted into that container.err-ignore" line points aterr-ignore(a file of regexes) as the thing not to reach for, where ci: breaking-change gate on api-description.yaml (oasdiff) #249's wording was an override label. If you agree with dropping the label (see the deviations above), a clause saying so belongs in that comment, since the comment is where the next person looks.COMPATIBILITY.mdand the workflow comment: it compares documented paths, and the spec documents the canonical/v2/request/...paths only, so dropping the/v2/irma/...alias handlers before shrink the promised surface: deprecate /v2/irma alias, drop dead format_version, unify x-postguard semantics #257's deprecation has run passes it. That stays a review rule.COMPATIBILITY.mdpointing at the YAML comment plus the same wording inapi_gate.rs's header went with the conflict resolution.actions/checkout@v6in the new file against@v4throughoutbuild.yml. Both resolve; worth aligning whenever you next touch either.api_gate.rsnow pins the action ref as well asfail-on/include-checks, because the ref is what selects the engine (v0.1.10isFROM tufin/oasdiff:v1.26.1) that the verdicts were measured against, so a bump used to leave the suite green while the verdicts stopped describing CI. And root.gitattributesnow marks the spec-text: the mutation anchors are multi-line raw strings, rustc normalises CRLF inside those butread_to_stringdoes not, so a Windows checkout failedevery_mutation_still_applieson a clean tree. CI is ubuntu/macos only and never saw it.Refs #249, #242. Part of #247 (workstream C).