orchestrator: codex is quota-walled until 2026-08-20, so both reviewers are one claude-opus-5 — restore review independence by model when the latch is armed

[melihucar]

Gate: bootstrap
Agent: claude

Found by retrospective 34 (orchestrator/logs/retro-34.md, Finding 1).

This is not #126. #126 asks for the quota latch to be re-probeable and
expiring, on the evidence that a previous firing was transient. Today's firing
is not transient and a re-probe would correctly re-latch. This issue is about
what the loop should do while the latch is right.

Evidence

codex hit a real quota wall at 2026-08-15T09:17:00Z, on #13's impl.1.
orchestrator/logs/13/codex.impl.1.log ends:

error: you've hit your usage limit. upgrade to pro (...), visit
https://chatgpt.com/codex/settings/usage to purchase more credits or
try again at aug 20th, 2026 11:42 am.

Five days out. review_diversity_lost has fired four times over the run
(2026-08-13T02:31:03Z, 2026-08-14T10:08:35Z, 2026-08-14T19:06:57Z,
2026-08-15T09:17:00Z).

REVIEWER_AGENTS = {1: "claude", 2: "codex"} (orchestrator/loop.py:990).
Once codex_ok() is false, resolve() (orchestrator/loop.py:232-237) sends
slot 2 down the identical final line as slot 1:

def resolve(agent: str, role: str, escalate: bool = False) -> tuple[str, str | None]:
    if agent == "codex" and codex_ok() and not escalate:
        return "codex", None
    if escalate:
        return "claude", ESCALATED_MODEL          # == "claude-opus-5"
    return "claude", MODELS.get(role, MODELS["implementer"])

MODELS["reviewer"] is claude-opus-5 and ESCALATED_MODEL is
claude-opus-5, so there is no code path that gives the two reviewers
different weights. review_stage() states the cost itself
(orchestrator/loop.py:1271-1273):

Cross-model on purpose: two instances of one model reviewing a diff behave
closer to one reviewer than to two. Once codex is out of quota both become
Opus, losing vendor diversity but keeping independent contexts.

Every issue merged since the latch armed was reviewed by two claude-opus-5
instances. Verified on disk, not inferred:

$ for i in 13 166 163 168 165; do ls orchestrator/logs/$i/ | grep -E 'review2\.'; done
# claude.review2.*.log for all five; no codex.review2.* anywhere

$ ls orchestrator/logs/*/codex.review2.*.log
# 11 issues in the entire run ever had a codex reviewer:
# 5, 8, 9, 10, 11, 20, 39, 72, 79, 80, 106

$ ls orchestrator/logs/*/[a-z]*.review2.*.log | sed 's|.*/||; s|\..*||' | sort | uniq -c
  69 claude
  48 codex

Blast radius on the queue, not just the reviewers

$ gh issue list --state open --limit 300 --json body \
    --jq '[.[] | .body // "" | capture("(?im)^\\s*Agent:\\s*(?<a>codex|claude|duel)").a]
          | group_by(.) | map({a:.[0], n:length}) | .[] | "\(.a)\t\(.n)"'
claude  72
codex   47
duel     5
$ # plus 13 open issues with no Agent line, which default to codex

65 of 137 open issues (47%) name an agent that cannot run for the rest of the
run. And because resolve("codex", role) and resolve("claude", role) now
return the same ("claude", MODELS[role]) pair, Agent: codex and
Agent: claude are indistinguishable — the planner's routing decision, the
retrospective's routing decision, and the whole duel alternation carry zero
information while the latch is armed. Nothing says so anywhere; the issue bodies
still read as if a vendor was chosen.

What to change

orchestrator/loop.py only. One behaviour: when the codex slot falls back, give
it a different model rather than a second copy of reviewer 1.

1. Add a constant next to MODELS (orchestrator/loop.py:99-107):

   # The stand-in for codex once the quota latch is armed. A DIFFERENT
   # generation, not the same weights twice: review_stage()'s own comment is
   # that two instances of one model behave closer to one reviewer than to two,
   # and that is the check the merge path rests on.
   FALLBACK_MODEL = os.environ.get("FR_MODEL_FALLBACK", "claude-opus-4-8")

2. In resolve(), return it when standing in for codex and not escalating:

   if agent == "codex":            # reached only when codex_ok() is False
       return "claude", FALLBACK_MODEL

   Keep the escalate branch ahead of it — escalation is a deliberate reach for
   the strongest model and must not be downgraded.

3. disable_codex() (orchestrator/loop.py:216-228) already records
   review_diversity_lost. Add the model actually substituted to that record —
   record("review_diversity_lost", reason=reason, fallback=FALLBACK_MODEL) —
   so a later retrospective can tell "lost vendor diversity, kept model
   diversity" from "lost both".

Acceptance criteria

• A test in orchestrator/check_orchestrator.py that pins the invariant
  directly: with _codex_disabled = True, resolve("claude", "reviewer") and
  resolve("codex", "reviewer") return different model strings. It must
  fail if either the constant is set equal to MODELS["reviewer"] or the new
  branch is deleted.
• A test that resolve("codex", "reviewer", escalate=True) still returns
  ESCALATED_MODEL — escalation outranks the fallback.
• bash scripts/gate.sh bootstrap passes, including orchestrator-runnable.

Out of scope

• Re-probing, expiring or clearing the latch — that is orchestrator: the codex quota latch is permanent, survives restart, and arms on one substring — 22 straight reviews with no vendor diversity and duel silently collapsed #126, and it is not the
  fix here: codex is genuinely walled until 2026-08-20.
• Changing REVIEWER_AGENTS, the number of reviewers, or the review verdict
  parsing.
• Rewriting the 65 open issue bodies that say Agent: codex. The fallback
  already routes them correctly; only the reviewer slot is losing something
  real. Do not touch other issues' bodies.
• Any model choice for the implementer, critic, fixer, planner, resolver or
  unblocker roles. Reviewer independence is the property under repair.

[melihucar]

Retro 35 — the diversity loss is now measurable in the verdict record, not just inferred from the latch.

review_verdicts carries both reviewers' decisions per round. Splitting the 86
scored rounds at the moment the codex latch armed (2026-08-15T09:17:00, after
which resolve() sends slot 2 down the same final line as slot 1):

	rounds	block/block	split (1-1)	pass/pass
pre-latch (claude + codex)	65	25 (38%)	21 (32%)	19 (29%)
post-latch (claude + claude)	21	9 (43%)	4 (19%)	8 (38%)

jq -R 'fromjson? // empty' orchestrator/logs/events.jsonl \
  | jq -r 'select(.event=="review_verdicts")
           | [(if .ts < "2026-08-15T09:17:00" then "PRE" else "POST" end),
              (.verdicts|to_entries|map(.value)|sort|join("/"))] | @tsv' \
  | sort | uniq -c

Two reviewers running the same model disagree at 19%, where a claude/codex
pair disagreed at 32% — the split rate fell by 40% relative, and the
unanimous-pass rate rose from 29% to 38%. That is the signature of two
correlated samples replacing two independent ones: fewer arguments, and more
diffs clearing both slots on the first round. The second reviewer is now
supplying substantially less new information than it used to.

I am not promoting this above fr:p1 on that evidence. It shows the review
is measurably weaker, but I cannot point at a merge that shipped a defect
because of it — the post-latch sample is 21 rounds, and both reviewers still
block 62% of rounds, so the path is degraded rather than open. Promotion should
follow a concrete escape, not a shifted distribution.

Two notes for whoever implements this:

• The 19% residual is a floor worth knowing: even two instances of one model at
  temperature disagree on roughly one round in five, so "both reviewers passed"
  was never worth as much as it reads.
• The table above is only computable because review_verdicts records per-slot
  decisions. The retrospective prompt never mentions that event (prompts: the retrospective's event list has drifted from record() — 9 emitted events it is never told to look for, 5 of them loop failures #101) — the
  measurement that justifies this issue is one no retrospective is told to run.

[melihucar]

Retrospective 38: demoting this to fr:p2.

Not because it is wrong — the latch is armed, review_diversity_lost has fired four times, and the analysis in the body holds. Because fr:p1 means on the critical path to the next milestone, and this is not.

Three reasons, in order of weight:

1. No evidence it is letting anything through. Both reviewers being one claude-opus-5 would matter if reviews had gone quiet. They have not: 32 review_block events today against 12 merges, and the most recent round (#169, 2026-08-15T15:34:20Z) was a {1:block, 2:block} on a real finding. Loss of vendor diversity is a degradation of review independence, not of review function, and nothing in the journal shows a defect surviving because of it.

2. The trigger expires on its own. orchestrator/logs/13/codex.impl.1.log says try again at aug 20th, 2026 11:42 am. Any run that reaches that date gets diversity back without this issue; any run that does not gets no benefit from fixing it late. The durable half of the problem — the latch being permanent and arming on one substring — is orchestrator: the codex quota latch is permanent, survives restart, and arms on one substring — 22 straight reviews with no vendor diversity and duel silently collapsed #126, and orchestrator: the codex quota latch is permanent, survives restart, and arms on one substring — 22 straight reviews with no vendor diversity and duel silently collapsed #126 is fr:p2. This issue should not outrank its own root cause.

3. It is the loop ranking its own housekeeping above the port. Filed by retrospective 34 at fr:p1, into a band orchestrator: priority sorts above rank_key's housekeeping term, so a retrospective promotion is the one way to undo the #23 fix — and 3 of 3 claimable fr:p1 issues are now fr:meta #202 measured as 100% fr:meta among claimable issues. With this demotion the claimable fr:p1 band is queue: 17 open server-lane issues cite files that exist on no branch — #13 merged a different layout than the bodies were written against #184 alone — 17 open server-lane issues cite files that exist on no branch — which is port damage and belongs there. That is the shape the band is supposed to have.

No change to the issue's content or its Gate:/Agent: lines.