Retry capacity rejections on a designated backup model, and refuse a review whose repository moved - #101
Merged
patriyang merged 12 commits intoAug 11, 2026
Conversation
#96: a new fake-codex `model-at-capacity` behavior refuses one model with the real server's wording before producing any item, so the same prompt succeeds on another model. A task run against it exits 1 with only `failureMessage: "Selected model is at capacity. Please try a different model."` — nothing machine-readable, and no fallback. #98: holds the detached worker at its process-start-time probe with a fake blocking `ps`, commits the dirty tree while it is held, then releases it. The pinned `working-tree` target is honored but its content has moved into the branch diff, so the worker hands Codex a prompt whose entire repository context is `(none)` four times over and the job completes clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
patriyang
deleted the
fix/96-98-capacity-fallback-and-review-state-drift
branch
August 11, 2026 18:46
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #96
Closes #98
#98 is the other half of the gap #97 half-closed, and builds on its
request.targetpinning. This branch was developed stacked on #97; #97 merged mid-flight, so it is now rebased directly ontomainand the diff is only this work.Both issues land together because they converge on one surface: a machine-readable failure class on the job record. #96 needs it to tell a transient capacity rejection from a real failure; #98 needs it to tell "the repo moved" from a review that genuinely found nothing.
#96 — capacity rejections were terminal, and controllers had to string-match
A capacity rejection is transient and content-independent, but it arrived as
status: failedwith the reason only in the prosesummary. Every controller had to recognize"Selected model is at capacity"by hand and re-dispatch.Three parts:
A failure class. New
lib/failure-class.mjsclassifies a turn's error message.runAppServerTurn/runAppServerReviewnow returnfailureClassandretryable;executeTaskRun/executeReviewRunput them on the payload;runTrackedJobpersists them on both the job file and the index, on the completion path and the throw path./codex:status --jsonand/codex:result --jsonexpose them, and the human output carries one terse line.A designated backup model — with nothing hardcoded. Model names and labelling conventions churn, so a pinned fallback would rot. Resolution is
CODEX_COMPANION_FALLBACK_MODEL(set it tononeto disable) →/codex:setup --fallback-model→ live discovery frommodel/list, preferring the advertised default and skipping the model that was at capacity. Step 3 is what survives renames. The existing inlinemodel/listwalk was extracted intolistAdvertisedModelsand shared with effort validation rather than duplicated; it takes an optional stopping predicate so effort validation keeps its early exit.A retry that cannot repeat work. Exactly one fallback attempt, on the same client and thread, changing only the model. It is gated on the turn having produced nothing.
That guard is the load-bearing part, and the first version of it was wrong.
recordItemonly recordsfileChanges/commandExecutionsonlifecycle === "completed", so a command that had started and not finished looked like "produced nothing" — and a retry would have re-run it. The guard now counts every item notification regardless of lifecycle. This is not hypothetical: the Codex implementer working on this branch took a capacity rejection roughly five minutes into a run, well after it had started editing.#98 — a queued review could silently review nothing
#97 pinned the resolved target's scope. It did not pin the repository state that scope resolves against. So:
--scope autoon a dirty tree pinsworking-tree, you commit before the detached worker starts, and the worker reviews a now-clean tree.Reproduced before fixing — the worker handed Codex a prompt whose entire repository context was:
and the job completed. A clean review of a change nobody looked at, which is the worst direction to fail in, because a clean review gets trusted.
Chosen fix: fail loudly on drift (option 1 of the three the issue laid out). Smallest change, keeps the current inputs, and converts a silent false negative into a visible retryable error. Snapshotting the diff at enqueue is the only fully deterministic answer and stays open — nothing here forecloses it.
captureRepoStateIdentityrecordsheadOid, plusbaseOidfor branch targets and a working-tree digest for working-tree targets.executeReviewRunrecomputes and refuses before anything can start a Codex turn, throwing an error carryingfailureClass: "state-drift"andretryable: true. Only background reviews capture an identity; the foreground path's window is a few statements and a check there would only produce spurious failures. Jobs persisted by an older plugin version have no identity and skip the check.Review rounds tightened the digest three times, each closing a real hole:
git status --porcelain=v2carries a blob OID for unstaged content. It does not — v2 gives HEAD and index OIDs only, so a further edit to an already-dirty tracked file was invisible. Dirty tracked paths now fold ingit hash-object.formatUntrackedFilefollows the link and reads the destination. It now mirrors what the review actually collects, bounds included.Submodules get bounded, deliberately non-recursive handling: a dirty tracked directory folds in that submodule's own HEAD and status digest, one level, no nesting.
Deep review
Three rounds, the cap for this repo. Each round found real defects; all were fixed except one I pushed back on and then reversed.
Round 1 (4 findings, all accepted). The worst was that
retryable: truewas reported for a capacity failure on a turn that had already done work — the internal retry correctly refused, but the machine-readable flag told external controllers the opposite.retryablenow means safe to repeat, judged against the turn actually returned, so it and the internal guard cannot drift apart. Also:/codex:reviewnever used the backup model at all (the README promised it did), submodule diff content was missing from the fingerprint, and a successful fallback still attributed its output to the model that had rejected it.Round 2 (2 findings, both accepted). A review is not always limited to a frozen prompt — native reviews and
self-collectmode (large diffs) have Codex read the repository live during the run, so a single check at worker start cannot catch state that moves mid-run. Added a post-run revalidation, scoped to exactly those two modes; an inline-diff review keeps its findings, since discarding a completed multi-minute review whose prompt was already frozen would be a costly false positive.Round 2 also re-raised the untracked-content hashing I had rejected in round 1. I had argued that if the rendered prompt is byte-identical then nothing reviewable changed. That was only true for inline-diff mode — in native and self-collect modes Codex reads raw bytes the digest never saw. Rejection withdrawn; untracked regular files are now hashed with
git hash-object --no-filters, decoupled from display limits.Round 3 (2 findings, both accepted, fixed directly at the review cap). The pinned-state check ran a few statements before
collectReviewContext, so it validated a moment rather than the content actually frozen into the prompt; it now runs immediately after collection. And the classifier matched a baretry a different model, which the server also appends to unrelated model-compatibility errors — that pattern is gone, since the real capacity message carriesat capacityanyway.Verification
npm test: 341 passing, 0 failing, run outside the Codex sandbox.npm run build(tscwithcheckJs): clean. It was failing with threeTS2339errors onturnState.error?.message— caught locally, not by review; those now route through the existingextractErrorMessage.SPEC_COMPLIANTand code-quality reviewAPPROVED, each after one fix round. Task 3's second round came back with no issues at any severity.387a9dd) and are unmodified.Notes
Two reviewer findings were rejected rather than implemented: an "edited test files" flag that contradicted an explicit later instruction, and an "extra work" flag against
8e50b9a, which is this branch's own Task 2 follow-up that the review range happened to span.Two known residuals, both deliberate:
git hash-objectprocess at a time, so a very large dirty tree makes--backgroundenqueue slower than it needs to be. Batching would fix it; it is not correctness.#99 was left unclaimed for a separate agent — it only touches
plugins/codex/commands/rescue.mdand shares no files with this branch.🤖 Generated with Claude Code