Repository navigation
fix: the scanner and sweep reported infected repos as clean, in seven different ways - #3
Merged
Merged
Conversation
org_sweep.sh had three independent ways to return "clean" on an infected repository. Each is fixed here with a regression test, because the previous round of this work shipped a test suite that had no assertions and always exited 0. Silent-clean bugs fixed: - The sweep called `rg` with no fallback and discarded its stderr. Where rg is only a shell function -- `command -v rg` succeeds in zsh and fails inside bash, which is the case on at least one engineer workstation -- the command failed, $hits came back empty, and every repo printed "HEAD clean (hard IOCs)". Reproduced against a known-infected fixture: the sweep called it clean. Replaced with git grep, which is also what makes ref-wide scanning cheap, plus a self-test that aborts rather than report anything as clean if a rule cannot match its own canary. - The campaign tag was grepped as the literal 'A9-3727'. That is the obfuscator.io build; the older _$_ shuffle build writes global['!']='9-3727-2' with no A prefix and was invisible to the sweep. Now matched as a pattern (global.i / global['!'] assignment of A?#-####-#), which also survives build-number rotation. Verified: the old pattern misses the no-prefix build, the new one catches both. - Whitespace padding was checked with `tail -c 2000`, but the padding sits at the START of the payload, so the last 2000 bytes are pure payload with no spaces at all. Now checked over the whole file at a 200-space threshold (observed at 507). This is not the rejected line-length heuristic: minification strips whitespace, so vendored bundles have long lines and no long space runs -- pinned by a negative-control test. Coverage gaps closed: - Enumerates every non-archived repo in both orgs via the API instead of 11 hardcoded names. Self-propagation targets any repo the victim can push to. - Clones --mirror and scans all refs, so refs/pull/* is covered. A normal clone does not fetch them, and poisoned blobs are still sitting there. - Adds the 'Sec-V' quoted key and bare AUTH_API_KEY to the signature set. The bare form is what sees the .env carrier; atob(process.env.* cannot. - Refs are scanned in batches so large repos do not blow the argument list. Reporting: - UNKNOWN is now distinct from CLEAN. A repo that failed to clone or whose engine errored has been proven nothing. Exit 1 infected, 2 not-scanned. - Ref counts, batch counts, clone failures and suppressions are all printed. A silent cap reads as coverage. - Verdicts are tallied through a file, not a shell variable: scan_repo runs in a subshell, so the previous counter was lost and the summary would have reported zero infected regardless of findings. - Token now comes from the environment or `gh auth token`, not argv, where it was visible to every user on the box via ps. Adds an allowlist that is scoped to paths, never to repositories, and that still prints what it suppresses -- the detection tooling matches its own indicators, so without it a nightly sweep over 355 repos is permanently red. test_org_sweep.sh: 12 assertions, all passing, including a negative control that fails against the previous implementation. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Same failure direction as the sweep fixes in the previous commit: the section
produced no output and the scan continued to a clean verdict.
The grep branch of section 1c spelled its alternation out by index,
"${REGEX_PATS[0]}|${REGEX_PATS[1]}|${REGEX_PATS[2]}|${REGEX_PATS[3]}"
against a three-element array. Under `set -u` that is an unbound variable, so
the subshell died before grep ran and the section reported nothing. It only
ever took the rg branch on hosts with a ripgrep BINARY -- and rg is frequently
just a shell function, so `command -v rg` succeeds in an interactive zsh and
fails inside this script, which is exactly the case on at least one engineer
workstation. Reproduced against a file carrying two of the three patterns:
section 1c reported no loader-family patterns.
The alternation is now built from the array with IFS, so it cannot drift out of
sync with its length again, and stderr is no longer discarded in that branch --
an engine failure prints instead of reading as "no findings".
Also fixes the third pattern. It was `Sec-V:`, but the bytes in the payload are
the quoted object key `'Sec-V':`, so the quote is part of the indicator: the
unquoted form matches zero real payloads. Verified both ways against a fixture.
test_rg_detection.sh gains three assertions for section 1c, including a negative
control pinning the quoting. They fail against the previous implementation and
pass against this one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Validated against the four byte-exact carriers on 2026-08-27. The set contains
THREE distinct build IDs, not one:
'9-3727-2' tailwind.config.js wave-1 shuffle
"A9-3727-2" karma.conf.js obfuscator.io
"A9-3727-3" eslint.config.mjs obfuscator.io
So the campaign tag is a build number that has already rotated, not a fixed
marker -- which is also why 'Sec-V' reads as the sole detector for the eslint
carrier. The marker is not absent there; it changed.
That matters because the wave-1 carrier is found by that number ALONE. It
carries no 'Sec-V' and no AUTH_API_KEY. Had it shipped as 9-3727-3, a
literals-only rule would have missed it completely.
The generalised marker already in this branch matches all three, including the
rotated one. This test pins that: it asserts first that no literal matches its
fixture, so the case cannot quietly stop proving anything, then that the marker
still fires.
15 assertions, all passing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This was referenced Aug 27, 2026
The sweep clones with --mirror, which fetches refs. An object reachable from no ref is never looked at. GitHub keeps serving those objects by SHA long after a squash orphans them, and four payload blobs were retrieved that way from repos this sweep had reported CLEAN -- ovh-k8s-cluster and support among them, the latter confirmed to be the whole variant C implant. The scan was not wrong. The word was. CLEAN reads as "the implant is gone"; what the run actually proves is "no ref in this repo reaches the implant". Those are different claims and the gap between them is where two poisoned repos got written off. So there is no CLEAN verdict any more: RESULT <repo> INFECTED|NO_REF_HITS|UNKNOWN <hits> and every non-abort exit path now prints what the run does not cover -- orphaned objects, which --deep does not reach either, since `git rev-list --all` still walks only what is reachable. They cannot be enumerated at all, so verifying them is a per-SHA check against the incident catalogue, not a scan. Saying that in the output is the point: no future reader should have to rediscover it. The caveat is a function called from both the org path and --local. Writing it inline in the summary left --local silently without it, which is the same failure in miniature -- a partial result read as a total one. The test caught that, not review. test_org_sweep.sh pins all three: the NO_REF_HITS token, the absence of any CLEAN verdict, and the caveat being printed. 18 assertions, all passing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The rules lived in polinrider_scan.sh and org_sweep.sh separately, plus a third
hardcoded copy inside classify_config(). They drifted: the sweep matched the
campaign tag as the literal 'A9-3727' long after the scanner had generalised it,
which left the sweep blind to the wave-1 variant on every run it ever performed.
Nobody noticed, because a rule that has drifted still reports CLEAN.
rules.sh is now the only place a rule is written down. Both scripts source it and
abort if it is missing.
It is not a flat list. The two tools have different false-positive surfaces and
that difference was deliberate, so the file carries three sets:
core rmcej%otb%, wuqktam... hard everywhere
repo-only 'Sec-V', helloipbot, hard in a repo sweep
AUTH_API_KEY, 9-3727-2
host-only atob(process.env.AUTH_API_KEY hard on a workstation
Flattening these would have promoted 'Sec-V' to a hard indicator on developer
machines, where it appears as quoted text in Claude Code transcripts -- a
recorded FP class that would produce false COMPROMISED verdicts.
Three things surfaced while doing this:
- Neither tool carried the bare 9-3727-2 literal. Both relied on the marker
pattern alone, which requires the global-assignment context to match. The
handoff lists that literal as load-bearing. It is now in the repo set.
- classify_config() held a third copy of the signatures, inline.
- The comment claiming the payload line carries ~5,000 padding spaces was wrong.
It is 507, measured on all three obfuscated carriers. The conclusion that
comment supported was right; the number was not.
test_rules_single_source.sh, 21 assertions: both sets non-empty, the shared
indicators in both, the deliberate split preserved in each direction, no script
defining a rule inline, both scripts sourcing and aborting without rules.sh, and
the marker matching every known build plus a rotated one it has never seen.
All five suites pass. Verified against the four byte-exact carriers: all four
detected, marker fires on all three obfuscated builds including A9-3727-3.
Known gap, recorded rather than solved: rules.sh is the single source for BASH
consumers. A non-bash consumer -- the PR-check service -- cannot source it. That
needs a canonical format both can read, decided when we know what the service is
written in. Do not solve it by copying the strings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…l match malware
org_sweep.sh has had an engine self-test since the rg-without-fallback failure.
polinrider_scan.sh never did, and it is the half that runs on developer
machines, where the environment is least predictable and the same failure is
waiting: `rg` is frequently only a shell function, so `command -v rg` succeeds
in an interactive zsh and the binary is absent inside this bash. The command
fails, output comes back empty, and empty reads as "nothing found".
Both scripts now plant known-positive carrier bytes, run their real rules with
their real flags, and abort rather than report anything clean if a rule does not
fire. There are negative controls too: a clean file must not match, and the
node_modules exclude must still apply -- a rule that fires on everything detects
nothing, and that failure is invisible without one.
canaries.sh, and why it is a separate file
This went through two versions that were both false, and neither failed loudly:
1. Fixtures generated from ${SIGS[@]}. That proves the rules match themselves.
It passes with every signature silently rewritten.
2. Real carrier bytes, hardcoded in rules.sh. One find-and-replace across that
file rewrote a signature AND the sample proving it worked, in the same
pass. The self-test kept passing on a rule that matched no real carrier.
Evidence that moves whenever the claim moves is not evidence. The samples now
live in canaries.sh, out of reach of an edit to rules.sh, and that separation is
the entire mechanism rather than a matter of tidiness. rules.sh sources it, so
consumers still have one thing to source.
Both defects were found by the sabotage test, not by reading the diff. That test
breaks one rule at a time in a throwaway copy and requires the scanner to refuse
to run; it is the only thing standing between this canary and a third false
version of it.
Also fixed while here: a sourced file that `return`s does NOT abort its caller,
so a missing canaries.sh let the scan proceed with the canary silently
evaporated. Both scripts now check on their own side and exit 2.
Five suites pass. The sweep's sigs.js fixture is kept as well -- built from the
rule set on purpose, since it proves the engine can match literals containing
quotes and $, which is a different claim from the rules being right.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This repository has no CI. `gh pr checks` reports zero checks on every open PR, so the four fixture suites run only when someone remembers to run them. That is a poor fit for what they guard. Every defect these suites pin fails in the same direction -- reporting an infected file or repository as CLEAN -- so a regression here does not break a build, it quietly stops finding malware. The repo has already been bitten by exactly this: test_config_classification.sh had no assertions at all and passed for weeks while testing nothing, which is recorded in f5bc89e. Suites are DISCOVERED (test_*.sh at the root), not listed. Listing them means a new suite silently never runs until someone edits this file, and it also breaks the other way: test_org_sweep.sh does not exist on main yet -- it arrives with PR #3 -- so a workflow naming it would fail on main for the wrong reason. The job errors out if it discovers zero suites, so "passed by finding nothing" cannot happen either. Runs on a matrix of both search engines. The scanner falls back to grep where no ripgrep BINARY exists, because `rg` is frequently only a shell function and `command -v rg` can succeed in an interactive zsh while failing inside the script. That exact difference hid a real detection failure, so the grep path needs to be covered rather than assumed. Also runs bash -n over every script. Syntax only: these are bash-specific and macOS bash-3.2 compatible on purpose, so a style linter would fight the code. actions/checkout is pinned by SHA, verified to resolve to v5.0.0. Verified locally: the three suites on main are discovered and pass. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Everything in this repo fails toward "clean". Seven defects were found in a single day — four in the tooling, three introduced while fixing it — and every one reported an infected file or repository as CLEAN. None ever produced a false alarm. That asymmetry is what this PR is written against.
Folded #5 and #6 in here so this is one review, not four.
1. Four ways a poisoned repo was reported clean
rgwith no fallback. Wherergis only a shell function — common;command -v rgsucceeds in an interactive zsh and fails inside the script's bash — the command failed,$hitscame back empty, and every repo printed "HEAD clean".${REGEX_PATS[0]}|…|${REGEX_PATS[3]}against a 3-element array, so underset -uthe subshell died before grep ran and the section printed nothing.9-3727-2,A9-3727-2,A9-3727-3across a four-file sample set. A literals-only rule would have missed the third. The marker is now the generalisedA?#-####-#pattern, pinned by a test against a build we have never seen.2. There is no CLEAN verdict any more
The sweep clones with
--mirror, which fetches refs. An object reachable from no ref is never looked at, and GitHub keeps serving those objects by SHA long after a squash orphans them. Four payload blobs were retrieved that way from repos this sweep had called CLEAN —ovh-k8s-clusterandsupportamong them, the latter confirmed to be the whole variant C implant.The scan was not wrong. The word was.
Every non-abort exit path now prints what the run does not cover.
--deepdoes not close the gap either —git rev-list --allstill walks only what is reachable. Orphaned objects cannot be enumerated at all, so verifying them is a per-SHA check against the incident catalogue, not a scan.The caveat is a function called from both the org path and
--local. Written inline in the summary it left--localsilently without it — the same failure in miniature, a partial result read as a total one. The test caught that, not review.3. One source of truth for the rules
The rules lived in
polinrider_scan.sh,org_sweep.sh, and a third hardcoded copy insideclassify_config(). They had drifted: the sweep matched the tag as the literalA9-3727long after the scanner generalised it, leaving it blind to variant A on every run it ever performed.Not a flat list — the two tools have different false-positive surfaces and that difference was deliberate:
corermcej%otb%,wuqktam…repo-only'Sec-V',helloipbot,AUTH_API_KEY,9-3727-2host-onlyatob(process.env.AUTH_API_KEYFlattening them would promote
'Sec-V'to a hard indicator on developer machines, where it appears as quoted text in Claude Code transcripts — a recorded FP class that would produce false COMPROMISED verdicts.4. Neither script will run unless its rules still match malware
The sweep has had an engine self-test since the
rgfailure. The host scanner never did, and it is the half that runs on developer machines. Both now plant known-positive carrier bytes, run their real rules with their real flags, and abort rather than report anything clean if a rule does not fire. With negative controls: a clean file must not match, and thenode_modulesexclude must still apply — a rule that fires on everything detects nothing, and that failure is invisible without one.canaries.shis a separate file, and that is the whole mechanism. This went through two versions that were both false, and neither failed loudly:${SIGS[@]}— proves the rules match themselves; passes with every signature silently rewritten.rules.sh— one find-and-replace rewrote a signature and the sample proving it worked in the same pass.Evidence that moves whenever the claim moves is not evidence. Both defects were found by the sabotage test, which breaks one rule at a time in a throwaway copy and requires the scanner to refuse to run.
Also fixed: a sourced file that
returns does not abort its caller, so a missingcanaries.shlet the scan proceed with the canary silently evaporated. Both scripts check on their own side and exit 2.5. CI
The repo had none. Its suites ran when someone remembered — in a repo whose own history includes a suite that had no assertions and passed for weeks while testing nothing. Suites are discovered, not listed, so a new one cannot be silently skipped. Both engines run, because the grep/ripgrep difference has hidden a real detection failure before.
Validation
Every rule here is checked against four byte-exact carriers supplied by the IR lead on 2026-08-27 (SHA256-verified, held outside any repo). The claims in the comments are measurements, not inferences from a write-up — including two corrections: the padding run is 507 bytes, not the ~5,000 the old comment claimed, and the payload starts at column 510 in two carriers and 536 in the third, so the rule anchors on the run length and never on a column.
Five suites, all passing.
Known gap, recorded rather than solved
rules.shis the single source for bash consumers. The PR-check service cannot source a bash file. That needs a canonical format both can read, decided once we know what the service is written in. Do not close it by copying the strings — that recreates exactly the drift this PR removes.