ci: sweep leftover dead trufflehog job from consumer workflows - #525
Merged
Conversation
|
hyperpolymath
added a commit
that referenced
this pull request
Jul 27, 2026
The a2ml/S4 secret-scanning scorecard required BOTH trufflesecurity/trufflehog AND gitleaks in a2ml/.github/workflows/secret-scanner.yml. #525 deliberately removed the redundant TruffleHog job estate-wide, so the check trufflehog grep started failing — surfacing as "a2ml/S4: claimed PASS but check exited 1" in build-scorecards --verify. That is the second, previously-masked failure in the "Registry + topology in sync" job (masked on main because the registry-drift step failed first). Updated S4 check/evidence/system to the post-#525 reality: gitleaks + rust/shell secret jobs. Verified the new check holds and the dashboard is current. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
hyperpolymath
added a commit
that referenced
this pull request
Jul 27, 2026
…e-TruffleHog a2ml/S4 scorecard (#525 fallout) (#531) Greens the required **`Registry + topology in sync`** check, which has been red on main and blocking every PR. That job had **two** failures — the first masking the second: **1. Registry drift (step 4).** `build-registry.sh --check` reported `REGISTRY.a2ml is stale`. Regenerated from a clean clone of `main`: only 7 `source_hash` values updated to match the tree, no entries added/removed, `TOPOLOGY.md` unchanged. This is the recurrence #381's pre-commit guard is meant to prevent (it reached main without the hook). **2. Stale scorecard `a2ml/S4` (step 6, previously masked).** Once the registry step passes, `build-scorecards.sh --verify` runs and fails: `a2ml/S4: claimed PASS but check exited 1`. The check still required `trufflesecurity/trufflehog` in `a2ml/.github/workflows/secret-scanner.yml`, but **#525 deliberately removed TruffleHog estate-wide**. Updated S4's `check`/`evidence`/`system` to the post-#525 reality (gitleaks + rust/shell secret jobs). Verified locally: `build-registry.sh --check` passes, the new S4 check holds (gitleaks present), dashboard is current. (`--verify`'s full run needs ripgrep/xmllint/jq — the env-preflight guard correctly declines to judge without them; CI has them.) Follow-up (#381 option 2): a CI job that auto-regenerates + commits the registry on drift would stop this reaching main again. --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
hyperpolymath
added a commit
that referenced
this pull request
Jul 27, 2026
Phase 3 (preventive) of the secret-scanner programme. Two gaps, **one shape**. ## 1. The secret scanner had no canary Every defect found in the 201-repo sweep was invisible in CI, because **a gate that cannot fail is indistinguishable from a gate that passes**: | Defect | Consequence | |---|---| | `continue-on-error: true` on gitleaks | scan found a **live Cloudflare Global API Key in a PUBLIC repo** and reported success anyway — for months, across ~200 repos | | `rust-secrets` grepped `./src` only | Cargo workspaces keep code in `crates/*/src`; `grep` exits 2 on a missing dir and the `if` read that as *clean*. **8 of 64 Rust repos (12.5%) never scanned at all** | | `const.*KEY.*=` missed `static AUTH_KEY` | found **only** because a canary planted 5 secrets and the job reported 4 | `tests/test_secret_scanner_canary.sh` plants known secrets and asserts the scanner still trips — **9 assertions**: ``` PASS 5 planted secrets in ./src -> BLOCKS PASS all 5 planted secrets reported (not just the first) PASS secret in crates/*/src -> BLOCKS after cutoff (no root ./src) PASS secret in crates/*/src -> advisory before cutoff PASS advisory run reports the finding and never claims a clean pass PASS env::var / URL / lookup / path / comment / pragma -> ALLOWED PASS literal secret + later env::var mention -> STILL BLOCKS PASS malformed cutoff -> REFUSES TO RUN PASS no Cargo.toml -> skips cleanly ``` Note assertion 2 counts findings rather than checking the exit code. **The `static AUTH_KEY` defect passed an exit-code assertion** — exit 1 only proves ≥1 was found, not that all were. It **extracts the step body from `secret-scanner-reusable.yml` at run time** and executes it. Not a copy: a copy silently drifts from what ships, which is how these defects survived review. ### Mutation tested A canary that cannot fail would be the very thing it guards against. Reintroducing each historical defect **kills** it: | Mutant | Result | |---|---| | `./src`-only scan | ✓ killed | | drop `static` from the KEY pattern | ✓ killed | | make the job unable to fail | ✓ killed | | over-broad `env::var` exemption | ✓ killed | | disarm the date guard | ✓ killed | **5/5.** ## 2. `tests/` was wired into no workflow `tests/test_check_trusted_base.sh` has existed for some time and **never executed in CI** — the same shape a third time: a test that never runs looks exactly like a test that passes. `self-test.yml` runs every `tests/*.sh`, so adding a test is now sufficient to have it enforced. **Fail-closed**: empty discovery fails the job rather than reporting a vacuous pass. ## Also: registry regeneration 7 stale `source_hash` entries accumulated from #522/#525 plus these new files.⚠️ **This is the fourth registry-drift repair today.** The drift keeps reaching `main` because `Registry + topology in sync` is **not a required status check**, while a phantom `Dependabot` context forces `--admin` on every merge — which bypasses all protection including the check that correctly failed on the PR. That ruleset fix is owner-only and tracked separately; **without it this repair will be needed again.** ## Verified - both test files pass locally (2/2) - canary 9/9 against the shipping reusable - 5/5 mutants killed - `scripts/build-registry.sh --check` exits 0 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Resolves #507