doctor: expectation flags default to the running process — make the self-satisfying default impossible

`doctor`'s expectation flags default to the running process — make the self-satisfying default impossible

Observed directly on the deployed binary:

$ claude-gatekeeper --version
claude-gatekeeper 1.5.1

$ claude-gatekeeper doctor -h
  -expected-binary string    Expected binary path (default: this executable)
  -expected-version string   Expected version stamp (default "1.5.1")

Both defaults are self-referential. -expected-binary defaults to this executable;
-expected-version to the running binary's own version, baked at build time. So doctor's default
question is "do the installed surfaces match ME?" — never "do they match what should be
installed?" Every surface points at that binary and reports that version, so everything matches
by construction, and DRIFT reads OK.

The binary default is the worse of the two: even a correct version stamp would be checked against
the wrong artifact.

Why this is the root cause of gatekeeper-claude#69, not a cosmetic default

The fleet ran 1.5.1 for nine days while v1.6.0 was Latest, and nothing reported it. doctor ran
throughout, reported OK, and was correct on the question it asked. A tool whose
expected-version defaults to its own version is structurally incapable of reporting that it is
stale. That is an instrument that cannot fail — not through a weak assertion, and not through never
running, but because its default configuration makes it self-satisfying.

The fix is mechanical, not documentary

#72 added --check-latest, which asks the honest question. The temptation — the one I recorded
and am now correcting — is to document that the scheduled unit must keep invoking that flag,
because the default path will go on answering the self-referential question.

That note is constitutional: it works only if every future editor of the unit reads it, and the
failure it guards against is precisely the case where they did not. It also leaves the instrument
defective and adds a second artifact that must be read to use the first one safely.

Proposed instead:

-expected-version and -expected-binary take no default, or default to an external
source — a release manifest, a packaged version file — never to the running process.
With no safe default available, prefer requiring the flag. A required flag moves the
guarantee from caller discipline into the tool's interface, so every caller gets the honest
question: the scheduled unit, a human debugging at 3am, a future lane copying the invocation, a
support script nobody has written yet.

(Framing and the required-flag argument: ventures-xo.)

Negative controls

Invoke doctor with no expectation flags → must not silently compare the binary to itself.
Either refuse, or compare against the external source.
Point -expected-version at a value differing from the running binary → must report the
mismatch (this already works; it is the control that passes while the default fails).
Plant a stale binary and invoke via the default path → must not report OK. This is the
control that would have caught the nine-day silence, and it is the one no existing test covers.

Scope

This changes a CLI default and is therefore a compatibility-affecting change for any existing
caller relying on the implicit comparison — worth enumerating those callers before landing. The
scheduled unit added in #72 already passes --check-latest explicitly and is unaffected.

Negative control RUN and RED — by an independent party, not predicted

This issue's control ("plant a stale binary, invoke via the DEFAULT path, must not report OK") was
specified when filed. It has now been executed by a second seat and observed failing, which is
the standard this lane holds everything else to: a control ships with its red observed, not
described.

Verbatim from the run:

-expected-binary string   Expected binary path (default: this executable)
-expected-version string  Expected version stamp (default "1.5.1")

claude-gatekeeper --version      → claude-gatekeeper 1.5.1
claude-gatekeeper doctor --json  → {"ok": true, "expected_version": "1.5.1", ...}

ok: true against a binary that is two releases stale (v1.7.0 is Latest; 1.5.1 is deployed).
Both defaults confirmed self-referential by a party who did not write the issue. The baseline is
demonstrated rather than asserted.

A THIRD self-weakening default, from the same run

-min-surfaces defaults to 1, while the scheduled unit passes 3 — and this host has exactly
3 recognized surfaces (claude-plugin, codex-global, grok-global).

So under the default, two of three surfaces could disappear entirely and doctor would still
report success. The unit's explicit 3 is the only thing making surface loss detectable, and it
is caller discipline again rather than an interface guarantee.

That makes three defaults on the same command, all weakening in the same direction:

flag	default	effect of the default
`-expected-binary`	this executable	compares the binary to itself
`-expected-version`	its own version	cannot report that it is stale
`-min-surfaces`	`1`	tolerates losing 2 of 3 enforcement surfaces

The pattern is the finding, not any single default. Each is individually defensible as
"convenient", and together they make the default invocation of a health tool structurally unable to
report the three things it exists to detect. Whatever fix lands should address the class rather
than the two flags this issue was opened for.

Scope note on a related observation

The same run noted the first enumerated surface sitting under a plugin cache path containing
1.3.1, older than the deployed 1.5.1. That is known and separately tracked — it is the
version-named-directory defect in gatekeeper-claude#69, where a manual deploy writes into the
active cache directory rather than creating one named for the version. Expected in the sense of
"filed", not in the sense of "correct".

Control execution and the min-surfaces observation: ventures-xo.

Full default enumeration: the class is real, and there is a fourth instance

Enumerated every flag default in this repo — origin/main @ 33ca0ae (v1.7.0), non-test, 25 registrations across 7 flag sets — to test whether the self-weakening defaults are a class or three separate lapses.

`doctor` — four permissive defaults, not two

flag	default	effect
`-expected-binary`	`""` → `os.Executable()` (main.go:280-284)	compares the binary to itself
`-expected-version`	`version`	cannot report itself stale
`-min-surfaces`	`1`	tolerates losing 2 of 3 enforcement surfaces
`-check-latest`	`false`	the staleness comparison does not run at all

-check-latest is the new one and the most consequential: with the default, PublishedVersionProbe is nil (main.go:286-292) and version_invariant is absent from the output entirely. Enabled against live state it is correct and complete — reports fail, published_latest 1.7.0, all three surfaces stale at 1.5.1, and independently flags the plugin path/binary mismatch (path 1.3.1 vs binary 1.5.1).

The capability that detects a stale enforcing binary already exists and works. It is opt-in.

The mechanism, and the control that proves it

These defaults do not lean the same way by coincidence. A default is chosen so the tool works with no arguments; for a checking tool, "works out of the box" means "produces a clean result on a healthy system" — and that question is answered by making the check weaker. Three independent decisions, one selection pressure (framing: ventures-xo).

The control is inside this same binary. verify-release sets -min-surfaces to 3 — same concept, strict — and verify-release refuses to run without a tag (main.go:187-197; executed: exit=2, usage). No bare invocation exists, so the pressure never applies, and the default came out strict.

In a checking tool, the bare invocation must be the strictest configuration, or must refuse to run.

doctor runs bare and is permissive on every axis; verify-release cannot run bare and is strict. One file, one author, opposite outcomes, and the discriminator is structural.

Why the fix must be class-level

Fixing the four flags leaves the fifth to arrive the same way. The ask on this issue is the rule above, applied to doctor:

-expected-version — derive the expectation from something outside this executable (published latest, or an explicit pin). Never default to self.
-expected-binary — same; the enforcing binary is discovered, not assumed to be the one running.
-min-surfaces — derive the expected count from the surfaces the install actually configured. Do not simply copy 3 from verify-release: a hardcoded constant is the right direction and will not move when a fourth surface arrives.
-check-latest — on by default, with an explicit opt-out for offline use. A staleness check that must be requested is not a check.
Where a strict default cannot be derived (no network, no pin), report unknown and fail — never ok: true.

Sharpening detail

Run bare, a build of this tool returns ok: false (stamp dev vs surfaces 1.5.1). The self-referential default only conceals staleness when the tool IS the deployed binary — the production configuration. The defect is inert everywhere it would be noticed and live in the one place it matters.

Also note files and surfaces are distinct: MinSurfaces compares len(report.Surfaces) (internal/inventory/inventory.go:179). Live host reports 4 files, 3 surfaces — the extra file is ~/.claude/settings.json with commands_recognized: 0, i.e. the file inflating the count is the one contributing nothing. Anyone setting the bar from the files array sets it wrong by one.

Provenance

Baseline red demonstrated by an independent runner (ventures-xo) rather than specified by this issue's author. This enumeration was run against origin/main @ 33ca0ae, and the --check-latest evidence against a build of that ref in an isolated temp path — no installed surface was modified (~/go/bin/claude-gatekeeper mtime unchanged). Scope is this repo only; gatekeeper-codex and gatekeeper-grok have their own flag sets and were not checked.

Acceptance criteria amendment — and a withdrawal of item 3 above

Two corrections from ventures-xo, both of which change the fix rather than decorate it.

1. WITHDRAWN: "derive the expected count from the surfaces the install actually configured"

That was item 3 in my previous comment and it reintroduces the class inside the fix for the class. Derived from what is installed, -min-surfaces becomes expect as many surfaces as I find — which always passes, for the same reason -expected-version defaulting to self always matches.

Replacement, and it is sharper than "declared, not observed": the expectation must be produced by a different act, at a different time, than the check. published_latest works because the release act wrote it. A surface manifest works only if the install act wrote it. An expectation the checking act computes for itself is self-reference regardless of how it is spelled.

Grounded cost, so this is not hand-waved: setup installs one harness per invocation (-harness, default claude; main.go:451-461) and keeps no cross-invocation record of what it has configured. A declared manifest therefore means setup gains durable state it does not currently have. That is real work, not a flag change, and this issue should say so rather than imply a one-liner.

2. NEW ACCEPTANCE CRITERION: the test must reproduce the tool-is-the-deployed-artifact configuration

From ventures-xo, and it explains the nine days better than "nobody looked":

A self-referential check is invisible to testing by construction, because testing necessarily runs it somewhere other than production, and production is the only configuration in which the tool is the artifact under test.

Confirmed here: run bare, a build of this tool returns ok: false (stamp dev vs surfaces 1.5.1). A test exercising doctor from a build shows the check working — green, correct-looking, confidence-increasing. Looking would not have found it.

Sharpening, from the code: the seams that make this package testable are the same seams that make this defect untestable.

type Options struct {
    Home, ExpectedBinary, ExpectedVersion string
    MinSurfaces int
    VersionProbe, LookPath, PublishedVersionProbe func(...)
}   // internal/inventory/inventory.go:25-33

Any test constructing inventory.Options directly must supply ExpectedVersion to control the case — and supplying it is precisely what the defect gets wrong. The defaulting lives in runDoctor (main.go:265-267, 280-284), not in the library, so no unit test against inventory can catch this class, however thorough.

Therefore the criterion, concretely:

The regression test must invoke doctor with no flags, as a binary stamped through the release path, whose installed surfaces point at that same binary, and it must fail before the fix.

Every injectable seam left at its default. If the test sets ExpectedVersion, PublishedVersionProbe, or -expected-version, it is testing the library and the hole survives the green suite — for the second time.

It is implementable with what already exists:

Options.Home (inventory.go:26) redirects the entire surface tree to a temp directory — falls back to os.UserHomeDir() only when unset (inventory.go:97).
-X main.version={{.Version}} (.goreleaser.yml:17, over var version = "dev" at main.go:51) stamps a build.

So: build with -ldflags -X main.version=<v> into a temp dir, write hook files under a temp HOME pointing at that binary, run it with no arguments, assert ok:false.

3. Boundary — this class is specific to CHECKING tools, and the fix must not over-sweep

setup -binary also defaults to os.Executable() (main.go:441-448). That default is correct — installing this binary is the whole intent of the command. Self-reference is not the defect; self-reference in the thing that establishes the expectation a check is graded against is the defect.

The rule holds its shape: in a checking tool, the bare invocation must be the strictest configuration, or must refuse to run. setup is not a checking tool and is out of scope for this issue.

Correction to the amendment above: "different act, different time" is the WRONG property

The formulation in my previous comment does not hold, and the defect walks through it.

Run -expected-version against it. The stamp comes from -X main.version={{.Version}} (.goreleaser.yml:17) — applied by the release act, at build time, long before any doctor invocation. Different act, different time. It passes my rule, and it is the defect this issue exists to remove.

The property is not who wrote the expectation or when. It is whether the expectation is derived from the artifact under test (ventures-xo). The stamp lives inside the binary whose version is in question, so expectation and subject share an artifact no matter which act produced it.

expectation	source	verdict
`published_latest`	release registry — a different artifact	correct
surface manifest	installer's file — a different artifact	correct
`expected-version`	the binary under test itself	defect — and passes "different act, different time"

The operational test, because "different artifact" is necessary and not sufficient

A manifest produced by saving doctor's own output on a known-good install is a different artifact whose content is entirely subject-derived — and that is a realistic way for someone to implement the -min-surfaces fix on this issue. It would satisfy "different artifact" and reintroduce the class.

An expectation is independent if and only if it would NOT change when the subject is wrong.

Applied to every candidate in this thread:

expectation	if the subject were wrong…	independent?
binary's own stamp	stamp is wrong the same way	no
count of surfaces currently found	count is wrong the same way	no
baseline saved from a prior `doctor` run	baseline captured the same degradation	no
published latest release	unchanged	yes
installer-written surface manifest	still declares what it installed	yes
plugin path version vs binary-reported version	path name unchanged; disagreement is the signal	yes

That last row is why the live 1.3.1-directory-holding-1.5.1-binary mismatch is detectable at all.

Residual cost, stated so it is not discovered later: independence protects against blindness, not against a wrong but independent expectation. An intentional version pin will now fail against published_latest. That is correct behaviour — an intentional pin should be declared — but it is a real change in what a clean run means.

Acceptance criterion, restated

Not "derive from the install" (withdrawn), not "a different act wrote it" (wrong). Every default this issue touches must come from a source that would still be right if the thing it is checking were wrong.

A test CATEGORY this repo does not have, and the grep that finds the class

The general form of the testing blindness (ventures-xo): any defect in default-value computation is invisible to a suite that practises dependency injection, because injection is precisely the act of not using the default — and it scales the wrong way, so the better the injection discipline, the more completely the class hides. A thoroughly-tested package is not weak evidence about its defaults; it is structurally zero evidence, and it feels like strong evidence.

Composed with the selection-pressure finding: the place a checking tool is most likely to be wrong — its defaults, chosen for bare-invocation success — is exactly where its suite is structurally unable to look. Two independent mechanisms, one blind spot.

Default computation needs a test at the COMPOSITION ROOT, not in the library.

Here the composition root is runDoctor (main.go:261-316) — the part everyone agrees is too thin to test.

And the class has a mechanical signature, verified across all 25 flag registrations in this repo:

# 1. non-literal defaults in the registration
grep -nE 'fs\.(String|Bool|Int)\(' main.go | grep -vE ', (""|true|false|-?[0-9]+|"[^"]*"),'
   → main.go:266   expected-version, default `version`      ← 1 hit, and it is a defect

# 2. empty-string defaults resolved in the handler body
grep -nE 'if \*[a-zA-Z]+ == "" \{|os\.Executable\(\)' main.go
   → main.go:280-281   expectedBinary → os.Executable()     ← the second defect
   → main.go:416, 442  setup/uninstall                      ← in-bounds, correct (see boundary above)

Two greps enumerate every self-derived default in the binary and find exactly the two this issue is about. That is cheap enough to be a CI check, and it is the mechanical layer under the constitutional rule.

Independence is FAILURE-SCOPED — and applying that finds a gap this repo does not cover

Corrected form (ventures-xo), replacing the two-place version in my previous comment:

An expectation is independent of a subject FOR A GIVEN FAILURE if it would not change when the subject has that failure. Three arguments, not two.

The same expectation pair flips verdict depending on which defect is being detected:

expectation vs subject	failure being detected	independent?
published registry vs binary stamp	stale deploy	yes — a stale binary does not move the registry
published registry vs binary stamp	bad release (right tag, wrong content)	no — both descend from the release act

So this issue must name, per default, which failure its expectation is independent for:

default	failure it exists to detect	independent expectation
`-expected-version`	enforcing binary is stale	published release registry
`-expected-binary`	doctor is inspecting the wrong binary	installer's record of what it installed
`-min-surfaces`	enforcement surfaces silently lost	installer-written manifest of what it configured
`-check-latest`	(not an expectation — an enablement)	n/a; must default on

The gap that falls out, verified in `internal/releaseverify/releaseverify.go`

verify-release emits nine checks — release, assets, checksum, archive, three version probes, doctor, three harness canaries. Reading them under the failure-scoped rule:

checksum is independent for a corrupted download — archive bytes vs checksums.txt, and a truncated or tampered transfer moves one and not the other.
checksum is NOT independent for a bad release. checksums.txt is published by the same release act as the archive (releaseverify.go:121-142). Publish wrong content and both agree, wrongly.

Verified: there is no source-derived verification anywhere in the package — no target_commitish, no commit sha comparison, no attestation, no rebuild (grep returns zero across the file). Every comparand is either a release artifact or an operator-supplied tag.

Nothing in this binary is independent for the bad-release failure. The instrument that would be is one derived from source: a rebuild from the tagged commit compared byte-for-byte, or a provenance attestation produced by an act other than the release.

That is out of scope for this issue and should be filed separately; it is recorded here because the failure-scoped rule is what surfaced it, and the two-place reading concealed it — under "is the expectation a different artifact?", checksum looks independent and is.

Credit where it is due: `verify-release` does NOT inherit doctor's defect

Worth stating explicitly, because the flag table invites the opposite inference. verify-release derives its expectation from the tag the invoker supplied:

expectedVersion := strings.TrimPrefix(opts.Tag, "v")            // releaseverify.go:162
…
doctorExit, _ := runDoctor(ctx, opts.PluginBinary, opts.HostBinary,
                           expectedVersion, opts.MinSurfaces)   // releaseverify.go:183

It passes the expectation explicitly, so its terminal doctor check never falls back to the binary's own stamp. Combined with refusing to run bare and -min-surfaces 3, it is the correctly-constructed instrument in this binary — the invoker's tag is a third party to both the binary and the registry.

A default is not what a caller passes. Reading the flag defaults alone would suggest the release verifier ends in a self-referential check; the call site says otherwise.

One usability consequence of the fix, worth landing with it

Once -check-latest defaults on, an intentional version pin will fail against the registry. That is the correct trade — a deliberate deviation should be declared rather than silently indistinguishable from drift — but the current message reads as a defect:

enforcing version 1.5.1 != published latest 1.7.0

It should distinguish drift from a declared pin, and say how to declare one. Otherwise the first operator with a legitimate pin gets a red check with no route to a clean run except turning the check back off, which reintroduces exactly what this issue removes.

Correction to the framing above: the bad-release gap is CATEGORICAL, not deferred

My previous comment filed the bad-release gap as "out of scope for this issue and should be filed separately." That understates it, and the understatement licenses the wrong next move (ventures-xo).

You cannot verify a release against itself: every comparand inside the release act shares its fate. Adding a tenth check inside verify-release multiplies artifacts that all descend from the same event, so their agreement carries no information about whether that event was correct.

"Out of scope" invites someone to add a check inside the tool and believe the gap is closing. No check inside the tool can ever cover it.
"Impossible without a source-derived input" tells them where to spend.

The instruments I named — a rebuild from the tagged commit, or an attestation produced by a different act — are a new input class, not more verification. That is a decision about what the project depends on, not a piece of work to schedule.

Filed with that framing as #74. This issue keeps only the doctor defaults.

Navigation Menu

Repository navigation

doctor: expectation flags default to the running process — make the self-satisfying default impossible #73

`doctor`'s expectation flags default to the running process — make the self-satisfying default impossible

Why this is the root cause of gatekeeper-claude#69, not a cosmetic default

The fix is mechanical, not documentary

Negative controls

Scope

Negative control RUN and RED — by an independent party, not predicted

A THIRD self-weakening default, from the same run

Scope note on a related observation

Full default enumeration: the class is real, and there is a fourth instance

`doctor` — four permissive defaults, not two

The mechanism, and the control that proves it

Why the fix must be class-level

Sharpening detail

Provenance

Acceptance criteria amendment — and a withdrawal of item 3 above

1. WITHDRAWN: "derive the expected count from the surfaces the install actually configured"

2. NEW ACCEPTANCE CRITERION: the test must reproduce the tool-is-the-deployed-artifact configuration

3. Boundary — this class is specific to CHECKING tools, and the fix must not over-sweep

Correction to the amendment above: "different act, different time" is the WRONG property

The operational test, because "different artifact" is necessary and not sufficient

Acceptance criterion, restated

A test CATEGORY this repo does not have, and the grep that finds the class

Independence is FAILURE-SCOPED — and applying that finds a gap this repo does not cover

The gap that falls out, verified in `internal/releaseverify/releaseverify.go`

Credit where it is due: `verify-release` does NOT inherit doctor's defect

One usability consequence of the fix, worth landing with it

Correction to the framing above: the bad-release gap is CATEGORICAL, not deferred

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

doctor: expectation flags default to the running process — make the self-satisfying default impossible #73

Description

doctor's expectation flags default to the running process — make the self-satisfying default impossible

Why this is the root cause of gatekeeper-claude#69, not a cosmetic default

The fix is mechanical, not documentary

Negative controls

Scope

Activity

jim80net commented on Aug 4, 2026

Negative control RUN and RED — by an independent party, not predicted

A THIRD self-weakening default, from the same run

Scope note on a related observation

jim80net commented on Aug 4, 2026

Full default enumeration: the class is real, and there is a fourth instance

doctor — four permissive defaults, not two

The mechanism, and the control that proves it

Why the fix must be class-level

Sharpening detail

Provenance

jim80net commented on Aug 4, 2026

Acceptance criteria amendment — and a withdrawal of item 3 above

1. WITHDRAWN: "derive the expected count from the surfaces the install actually configured"

2. NEW ACCEPTANCE CRITERION: the test must reproduce the tool-is-the-deployed-artifact configuration

3. Boundary — this class is specific to CHECKING tools, and the fix must not over-sweep

jim80net commented on Aug 4, 2026

Correction to the amendment above: "different act, different time" is the WRONG property

The operational test, because "different artifact" is necessary and not sufficient

Acceptance criterion, restated

A test CATEGORY this repo does not have, and the grep that finds the class

jim80net commented on Aug 4, 2026

Independence is FAILURE-SCOPED — and applying that finds a gap this repo does not cover

The gap that falls out, verified in internal/releaseverify/releaseverify.go

Credit where it is due: verify-release does NOT inherit doctor's defect

One usability consequence of the fix, worth landing with it

jim80net commented on Aug 4, 2026

Correction to the framing above: the bad-release gap is CATEGORICAL, not deferred

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions

`doctor`'s expectation flags default to the running process — make the self-satisfying default impossible

`doctor` — four permissive defaults, not two

The gap that falls out, verified in `internal/releaseverify/releaseverify.go`

Credit where it is due: `verify-release` does NOT inherit doctor's defect