Repository navigation
ci: run the fixture suites on every change - #5
Closed
fabrizzio-dotCMS wants to merge 1 commit into
Closed
fabrizzio-dotCMS wants to merge 1 commit into
fabrizzio-dotCMS wants to merge 1 commit into
Conversation
This repository has no CI. `gh pr checks` reports zero checks on every open PR, so the four fixture suites run only when someone remembers to run them. That is a poor fit for what they guard. Every defect these suites pin fails in the same direction -- reporting an infected file or repository as CLEAN -- so a regression here does not break a build, it quietly stops finding malware. The repo has already been bitten by exactly this: test_config_classification.sh had no assertions at all and passed for weeks while testing nothing, which is recorded in f5bc89e. Suites are DISCOVERED (test_*.sh at the root), not listed. Listing them means a new suite silently never runs until someone edits this file, and it also breaks the other way: test_org_sweep.sh does not exist on main yet -- it arrives with PR #3 -- so a workflow naming it would fail on main for the wrong reason. The job errors out if it discovers zero suites, so "passed by finding nothing" cannot happen either. Runs on a matrix of both search engines. The scanner falls back to grep where no ripgrep BINARY exists, because `rg` is frequently only a shell function and `command -v rg` can succeed in an interactive zsh while failing inside the script. That exact difference hid a real detection failure, so the grep path needs to be covered rather than assumed. Also runs bash -n over every script. Syntax only: these are bash-specific and macOS bash-3.2 compatible on purpose, so a style linter would fight the code. actions/checkout is pinned by SHA, verified to resolve to v5.0.0. Verified locally: the three suites on main are discovered and pass. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Member
Author
|
Folded into #3 — commit |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
This repo has no CI.
gh pr checksreports zero checks on every open PR, so the fixture suites run only when someone remembers to.That is a poor fit for what they guard. Every defect these suites pin fails in the same direction — reporting an infected file or repository as CLEAN. A regression here does not break a build; it quietly stops finding malware.
The repo has already been bitten by exactly this.
test_config_classification.shhad no assertions at all and passed for weeks while testing nothing — recorded inf5bc89e:What it runs
Fixture suites (grep)test_*.sh, no ripgrep installedFixture suites (ripgrep)test_*.sh, ripgrep installedShell syntaxbash -nover every*.shBoth engines are in the matrix on purpose. The scanner falls back to grep where no ripgrep binary exists, because
rgis frequently only a shell function —command -v rgcan succeed in an interactive zsh and fail inside the script. That exact difference hid a real detection failure (section 1c returning nothing while the scan reported clean), so the grep path has to be covered rather than assumed.Suites are discovered, not listed
Two failure modes avoided:
test_org_sweep.shdoes not exist onmainyet — it arrives with fix: the scanner and sweep reported infected repos as clean, in seven different ways #3 — so a workflow naming it would fail onmainfor the wrong reason. Discovery makes this PR mergeable independently of fix: the scanner and sweep reported infected repos as clean, in seven different ways #3, and it picks that suite up automatically once fix: the scanner and sweep reported infected repos as clean, in seven different ways #3 lands.The job errors if it discovers zero suites, so "passed by finding nothing" cannot happen either. That is the same class of bug as everything else in this repo: a check that quietly succeeds by doing nothing.
Notes
actions/checkoutis pinned by SHA, verified to resolve to v5.0.0.POLINRIDER_DNS_TIMEOUT=1bounds the scanner's unified-log pass. It is a no-op on Linux but must not be able to stall a runner; it is a documented 120s default that made a fixture run look hung.bash -nonly. These scripts are bash-specific and macOS bash-3.2 compatible by design, so a style linter would fight the code rather than help it.Verified locally: the three suites currently on
mainare discovered and pass.Follow-up, not in this PR
Once this is green, the three checks are candidates for required status checks on
main— which is the point of having them.🤖 Generated with Claude Code