HKS-01..05 false negative when the scan root is itself a .claude directory

[marcelo-lima-clickbus]

Check ID

HKS-01

What went wrong

False negative — check failed but the artifact is actually there

Repository / minimal reproduction

mkdir -p repro/.claude && cd repro
printf '# Repro\n' > CLAUDE.md
cat > .claude/settings.json <<'EOF'
{
  "hooks": {
    "PreToolUse": [
      { "matcher": "Bash", "hooks": [{ "type": "command", "command": "echo gate" }] }
    ],
    "PostToolUse": [
      { "matcher": "Edit", "hooks": [{ "type": "command", "command": "echo feedback" }] }
    ]
  }
}
EOF

npx harness-score .          # Hooks & Guardrails: 14/14
npx harness-score .claude    # Hooks & Guardrails:  0/14

Expected vs. actual

Same settings.json, two scan roots, opposite results.

Scan root	Hooks & Guardrails
repro/	14/14
repro/.claude/	0/14

Scanning the .claude directory as the root fails HKS-01 through HKS-05. The config is valid, has PreToolUse (gate) and PostToolUse (feedback), and passes every check when scanned one level up.

Cause is in packages/cli/src/harness/registry.ts:88:

{ toolId: 'claude-code', kind: 'hooks', pathRegex: /(^|\/)\.claude\/settings\.json$/ }

The pattern requires a literal .claude/ segment. When .claude is the scan root, paths are relative to it and the file is just settings.json, which never matches:

NOMATCH  settings.json
MATCH    .claude/settings.json
MATCH    some/nested/path/.claude/settings.json

There is a second-order effect worth noting. readNormalizedHooks (harness/hooks.ts:153) picks whichever collected config declares the most events. With the root file excluded from collection, any unrelated nested .claude/settings.json (a vendored repo, a cached job artifact, a test fixture) becomes the only candidate and wins by default. In my case the reported evidence pointed at a scratch directory buried several levels down while the real config sat at the root, so the failure message named the wrong file rather than reporting nothing found.

A repo-root settings.json would fix the primary case, though it may be worth deciding whether a nested .claude/ inside vendored or cached content should be a scoring candidate at all.

harness-score version

1.5.2 (built from main at clone time)

[marcelo-lima-clickbus]

Follow-up on my own report: the same root cause hits three more specs, not just hooks.

All four Claude Code entries in harness/registry.ts require a literal .claude/ path segment:

Line	Kind	Pattern
68	skills	/(^|\/)\.claude\/skills\/[^/]+\/SKILL\.md$/
74	commands	/(^|\/)\.claude\/commands\/[^/]+\.md$/
83	subagents	/(^|\/)\.claude\/agents\/[^/]+\.md$/
88	hooks	/(^|\/)\.claude\/settings\.json$/

When .claude itself is the scan root, paths are relative to it, so the files are skills/x/SKILL.md, commands/x.md, agents/x.md and settings.json. None match.

That makes SKL-03 a false negative in exactly the same way HKS-01..05 are:

NOMATCH  commands/deploy.md
MATCH    .claude/commands/deploy.md

Reproduction, extending the one in the issue body:

mkdir -p repro/.claude/{commands,agents,skills/demo} && cd repro
printf '# Repro\n' > CLAUDE.md
printf '# Deploy\nSteps.\n' > .claude/commands/deploy.md
printf -- '---\nname: helper\ndescription: x\n---\nBody.\n' > .claude/agents/helper.md
printf -- '---\nname: demo\ndescription: x\n---\nBody.\n' > .claude/skills/demo/SKILL.md
# plus the .claude/settings.json from the issue body

npx harness-score .          # Skills & Commands 15/17, Hooks 10/14
npx harness-score .claude    # Skills & Commands  0/17, Hooks  0/14

Same content, 31 points apart, depending only on where the scan starts.

Scanning a .claude directory directly is not exotic: a dotfiles repo whose root is ~/.claude is a common way to version Claude Code config, and it is the case that led me here. My repo has 76 skills, 3 agents and 2 commands that the scanner cannot see.

One option is to also accept the root-relative form for these four kinds. Whether that is worth doing depends on how you want to treat a config directory as a scan target, which is your call.

[marcelo-lima-clickbus]

One more data point, and I think it reframes the issue: the same harness scores L1 · Documented (73/108) or L4 · Self-correcting (92/108) depending only on which directory you point the scanner at. Same files, same commit, four levels apart.

Scan root	Level	Score	Skills	Hooks
~/.claude	L1 · Documented	73/108 (68%)	0%	0%
the parent directory	L4 · Self-correcting	92/108 (85%)	100%	71%

The reason is the one already described in this issue. From the parent, .claude/ is a subdirectory and every path matches, so the scanner sees the 63 skills, the commands, the agents and the hooks config. From inside ~/.claude, the same content is invisible to SKL-*, AGT-* and HKS-*.

Both are legitimate ways to run this. I keep my Claude Code config as a git repo, which makes ~/.claude the natural scan root. But I launch Claude Code from my home directory, which makes the parent the natural project root. Neither choice is wrong, and nothing in the tool tells a user that this decision moves them four levels on the ladder.

That is what makes it worth fixing beyond the false negatives: the score stops being a property of the repository and becomes a property of where you happened to cd.

One option is to detect when the scan root is a tool config directory (a root named .claude, .cursor, .windsurf) and resolve the artifact paths relative to it. That would make both invocations agree.

---

Unrelated to the path matching, and possibly worth its own issue: scanning from a home directory hits MAX_FILES = 20000 (scan.ts:41) almost immediately. Mine has 291,063 files outside the SKIP_DIRS set, so the scan covers about 7% of the tree and stops. It does print the "results may be incomplete" warning, but it still prints a score and a maturity level next to it, which reads as a verdict rather than a partial sample. A truncated scan arguably should not report a level at all.

[marcelo-lima-clickbus]

A third failure mode, unrelated to path matching: HKS-02 rejects valid Claude Code hook events as unknown.

CLAUDE_KNOWN_EVENTS in harness/hooks.ts:45 lists five names:

const CLAUDE_KNOWN_EVENTS = new Set(['PreToolUse', 'PostToolUse', 'Notification', 'Stop', 'SubagentStop']);

The current documentation at https://code.claude.com/docs/en/hooks.md lists 31 valid events. The five above are a subset. Among the missing ones are SessionStart, SessionEnd, UserPromptSubmit, PreCompact, PostCompact, SubagentStart, PermissionRequest, TaskCompleted and FileChanged.

Reproduction, isolating a single key:

# .claude/settings.json with PreToolUse + PostToolUse only
npx harness-score .     # Hooks & Guardrails 14/14

# same file, adding a SessionStart handler
npx harness-score .     # Hooks & Guardrails 12/14
#   HKS-02 → Unknown event name(s): SessionStart

The penalty inverts the check's intent. HKS-02 exists to catch typos that fail silently, but here it penalizes a repository for registering a documented event, and the remediation text tells the user to remove a hook that works.

The Cursor list in the same file has 21 entries and looks current, so this reads like the Claude Code side simply predates the newer events rather than a design decision.

One option that avoids chasing the upstream list: keep the known set for the "did you typo this" signal, but treat an unrecognized name as informational instead of a failed check. A repository should not lose points for using a hook the scanner has not caught up with yet.