Skip to content

oasdiff gate fails open: fail-on ERR misses WARN-rated breaking changes #202

Description

@rubenhensen

The oasdiff gate landed in #201 with fail-on: ERR and no include-checks. Work on the pg-pkg equivalent (encryption4all/postguard#249 / #269) established that this setting fails open on several changes COMPATIBILITY.md forbids, because oasdiff rates them WARN rather than ERR:

  • removing or renaming an optional response property
  • removing a request parameter
  • the constraint-narrowing *-set family

Plus two checks that rate ERR but are opt-in, so they never run unless named: response-non-success-status-removed (401 → 403) and response-property-enum-value-removed.

This spec is less exposed than pg-pkg's — it marks far more fields required (34 required: occurrences vs pg-pkg's one), and a required property's removal does rate ERR — but the optional-property and request-parameter gaps apply to any spec, and cryptify's routes are unversioned, so there is no /v3 escape hatch: a break here breaks pinned clients immediately.

Do the measurement before changing the setting. The pg-pkg work found exactly one WARN check that fires on something the compatibility rules do not forbid (response-property-enum-value-added), and resolved it by making a new response enum value explicitly non-additive in COMPATIBILITY.md. Repeat that for this spec rather than assuming the same answer: mutate the spec each way, record which verdict each setting gives, and only then flip fail-on and add include-checks.

Reference implementation and its reasoning: encryption4all/postguard/.github/workflows/api-diff.yml and pg-pkg/tests/api_gate.rs, which pins the verdicts as an executable record. An equivalent test here would be worth having — this repo has no tests/ dir, so it would go inline per the repo's convention.

Part of encryption4all/postguard#247 (workstream C).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions