Skip to content

A type: adr document outside docs/adr/ escapes both gates: the corpus carve-out is content-based, the adr-smadr job is directory-scoped #209

Description

@zircote

Summary

The ADR carve-out added in #208 removes documents from the mif-validate corpus on a content predicate, but the job that is supposed to gate them instead is scoped to a directory. The two do not cover the same set, so a type: adr document that lands outside docs/adr/ is validated by nothing.

Where

scripts/lib/corpus.mjslistL3Docs() filters on isAdrDoc(), which tests frontmatter content across every entry in L3_DIRS:

export const L3_DIRS = ["docs/adr", "docs/architecture", "docs/runbooks", "docs/reference", "docs/explanation"];
...
return files.filter((f) => !isAdrDoc(f));

The file's own comment states the intended contract:

a type: adr document anywhere in the gated trees is owned by the structured-madr Action (the adr-smadr CI job)

.github/workflows/ci.yml — the adr-smadr job does not honour that contract; both new steps are pinned to one directory and one non-recursive pattern:

  mode: smadr
  path: docs/adr
  pattern: '*.md'

Escape path

Add docs/architecture/0006-some-decision.md with type: adr in its frontmatter:

  1. listL3Docs() drops it, so it is absent from listGatedDocs() and never reaches mif-validate (nor engine-parity.mjs or provenance-corpus-check.mjs).
  2. adr-smadr only walks docs/adr, so it never sees the file either.

Result: a document that claims MIF L3 conformance is gated by nothing, and CI is green. This is the same silent-coverage-drift class the corpus module was created to close (#32/#34) — the module's header comment is explicit that hand-synced lists disagreeing silently is the failure it exists to prevent.

A second, narrower variant of the same mismatch: listL3Docs() globs ${dir}/**/*.md (recursive), while the adr-smadr steps pass pattern: '*.md'. Whether a nested docs/adr/sub/0007.md is also uncovered depends on the structured-madr Action's globstar semantics, which was not resolved here — worth confirming rather than assuming either way.

Why this is not just theoretical

Nothing in the tree prevents it. tests/corpus.test.mjs pins the opposite direction (a non-adr doc under docs/adr stays gated) but asserts nothing about a type: adr doc under the other four L3 trees, and no gate rejects one.

Options (a design call, not a mechanical fix)

Three plausible resolutions, each with a different consequence:

  1. Narrow the carve-out to docs/adr/ — directory-based filter. Simplest and symmetric with the CI job, but loses the content-based property Make the five docs/adr ADRs conform to the adr genre and route them to the structured-madr gate #208 deliberately chose, and a type: adr doc under docs/adr becomes the only recognised shape.
  2. Fail closed in listL3Docs() — throw when a carved-out type: adr document is found outside docs/adr/. Keeps the content-based carve-out, makes the module's comment true, and matches the fail-closed discipline already in the file (missing tree, empty result). Costs an explicit error on an arrangement that may be legitimate.
  3. Drive the adr-smadr step from listAdrDocs() — emit the carved-out list and hand it to the Action, so the gate is by construction the exact complement of the corpus. Most faithful to the stated contract; requires the Action to accept an explicit file list rather than a path + pattern.

Acceptance criteria

  • Ubiquitous: a type: adr document under any L3_DIRS tree SHALL be validated by exactly one gate, with no arrangement that leaves it validated by none.
  • Ubiquitous: scripts/lib/corpus.mjs's stated contract for the carve-out SHALL match what .github/workflows/ci.yml actually gates.
  • Event-driven: WHEN a type: adr document is added outside docs/adr/, a PR-gated check SHALL fail or the document SHALL be validated — never silently pass ungated.
  • Ubiquitous: the resolution SHALL be covered by a test that fails on the current tree and passes after.

Found during code review of #208; not a regression from that PR's stated goal, and not blocking it.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: ci-cdCI/CD pipelines, GitHub Actions, or deployment automationstatus: needs-triageAwaiting initial review and prioritizationtype: bugSomething is not working as expected

    Type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions