HYP-S: delegate language-ban + sha-pinning criteria to live scanners (coverage 66→73%) - #624
Merged
Merged
Conversation
Track 1 coverage increment. The oracle now runs hypatia's offline content
scanners once per score (delegated_index/1 -> CLI.collect_findings/2, newly
public) and derives authoritative verdicts for the criteria whose file-presence
proxy was wrong or absent:
* language bans 5.1.1/5.1.2/5.1.3/5.1.5 -> cicd_rules/banned_language_file,
keyed by file extension. This is CARVE-OUT-AWARE (a .ts in an approved
path emits no finding), which the previous no_ext/1 file-presence check
could not honour — and it adds TS/ReScript, which were :unverified.
* sha-pinning 4.1.3 -> workflow_audit/{unpinned_action,wrong_sha_pin}
(previously :unverified).
Mapping is keyed on OBSERVED (module,type) finding keys, not the SSOT's
aspirational `detect` strings — ground-truthing showed they disagree (sha
pinning fires as workflow_audit/*, not supply_chain/*; all language bans
collapse to one banned_language_file type distinguished by extension). A scan
failure degrades to no delegated verdicts (criteria fall back to the
file-presence tranche / :unverified) — never a false pass, never a crash.
Honest ceiling: literal 100% offline coverage is not achievable — OpenSSF
Scorecard (4.2.x) is intrinsically external and licence (7.1.1) is flag-only.
Those stay :unverified, so scorecards stay provisional; firm tier claims will
require an accepted external attestation, a spec decision for RSR v2.0.x (§10).
Documented in the moduledoc.
Verified (Elixir 1.17.3/OTP 25): 12 oracle tests green incl. 3 new delegation
tests (planted .py fails 5.1.1; unpinned action fails 4.1.3; clean tree passes
the bans); rules+a2ml domain 194 tests, same 3 pre-existing failures. Live
corpus coverage 66% -> 73%; standards correctly drops (unpinned actions now
fail 4.1.3). Warning-clean; format clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0112RkdER2wtwHdNmbEhThUz
hyperpolymath
marked this pull request as ready for review
July 17, 2026 06:19
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Track 1 coverage increment toward RSR v2.0 ratification. The oracle now delegates the criteria whose file-presence proxy was wrong or missing to hypatia's own live scanners, run once per score.
What changed
delegated_index/1runsHypatia.CLI.collect_findings/2([:cicd_rules, :workflow_audit]— offline, no network) once and indexes the findings;collect_findings/2promoted fromdefptodefwith a doc note (the oracle shouldn't reimplement the per-file scan driver).5.1.1/5.1.2/5.1.3/5.1.5→cicd_rules/banned_language_file, keyed by file extension. This is carve-out-aware — a.tsin an approved path emits no finding, which the oldno_ext/1file-presence check could not honour — and it adds TS/ReScript, previously:unverified.4.1.3→workflow_audit/{unpinned_action,wrong_sha_pin}(previously:unverified).Ground-truth mattered
The SSOT's
detectstrings are aspirational and disagree with reality — I probed the live scanners rather than trust them: sha-pinning fires asworkflow_audit/*(notsupply_chain/*), and all language bans collapse to onebanned_language_filetype distinguished by extension. Mapping is keyed on the observed(module,type). A scan failure degrades to no delegated verdicts (fall back to file-presence /:unverified) — never a false pass, never a crash.The honest ceiling (a spec decision for you)
Literal 100% offline coverage is not achievable, by design — and forcing it would be overclaiming:
4.2.1/4.2.2is intrinsically external (needs the Scorecard service/API);7.1.1is flag-only (:review, never a pass/fail gate here).These stay
:unverified, so scorecards stayprovisional, so firm (non-provisional) tier claims are blocked until RSR v2.0 decides how to admit an external attestation (e.g. accept a committed.machine_readable/scorecards/result, or redefine coverage as "offline-automatable criteria"). That's a §10 spec choice — flagged, not silently resolved. Everything offline-automatable that maps cleanly is now delegated.Evidence (live corpus, real SSOT)
Coverage 66% → 73%;
standardscorrectly drops 61.6→58.9% (its unpinned actions now fail4.1.3); language-ban verdicts are now authoritative (carve-out-aware).Verification (Elixir 1.17.3 / OTP 25)
12 oracle tests green — 3 new delegation tests (planted
.pyfails5.1.1; unpinned action fails4.1.3; clean tree passes the bans). rules+a2ml domain 194 tests, same 3 pre-existing failures (2 vlang, 1 chapel). Warning-clean;mix format --check-formattedclean.Guardrails
Licence
7.1.1stays flag-only/:unverified— never delegated to a pass/fail here.collect_findings/2is called with offline modules only.🤖 Generated with Claude Code
https://claude.ai/code/session_0112RkdER2wtwHdNmbEhThUz
Generated by Claude Code