Skip to content

Surface a force-model pin that could not be served - #1111

Merged
munir-weave merged 2 commits into
mainfrom
munir/surface-dropped-force-model-pin
Aug 28, 2026
Merged

Surface a force-model pin that could not be served#1111
munir-weave merged 2 commits into
mainfrom
munir/surface-dropped-force-model-pin

Conversation

@munir-weave

@munir-weave munir-weave commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Problem

A user's /force-model pin is dropped for the turn when its provider isn't in EnabledProviders, the model is excluded, or it can't carry an image (turnloop.go). Dropping it is correct — dispatching to a provider the request can't authenticate to would 401. But the drop was completely silent: no log line, no marker. The turn fell through to the scorer and served a different model while the user was still looking at force-model applied: claude-opus-5 (anthropic) from the previous turn.

Reported from Codex desktop: a coworker pinned an Anthropic model, got the success ack, then checked GCP/router logs and found an OpenAI model serving every subsequent turn.

Two things made this hard to see:

  • forcedModelBinding validates the org allowlist, installation exclusions, gateway bindings, and policy-excluded providers — but not the enabled-provider set the next turn applies. So validation and enforcement disagree, and the ack is written a turn before the conflict surfaces.
  • The pinFound = false reset discards the pin with no trace. The maxed-out guard immediately below it does log its drop ("Session pin maxed out on previous turn"), which is the precedent this follows.

Change

  • Log the drop with a drop_reason (provider_not_enabled / excluded / not_image_capable).
  • Record it on turnLoopResult and render a routing marker naming the pin that didn't apply: ✦ **Weave Router** → gpt-5.5 · your force-model pin could not be served this turn (claude-opus-5).

The marker branch runs before the same-model suppression gate. That gate compares against the prior served model, so it would otherwise hide exactly the turns where the pin quietly stopped applying. Suggestion mode and empty decisions still suppress as before.

No routing behavior changes — this only makes an existing silent fallback observable.

Testing

  • TestService_SessionPin_ForcedPinDropped_SurfacesInMarker — end-to-end: an Anthropic-bound forced pin with only OpenAI wired falls through to the scorer, serves gpt-5.5, and surfaces the dropped pin.
  • Two routingMarkerFor cases: shown even when the served model is unchanged; suppressed in suggestion mode.
  • Both fail without the fix (verified by reverting the marker branch), and go test ./... passes with it.

Session-key derivation — investigated, not a live bug

A candidate second cause was DeriveSessionKey hashing FirstUserMessageText(): if /fm … is the first user message, the command is stripped from the body, so the next turn would derive a different session key and miss the pin. I wrote the unit test and it does reproduce (two different 16-byte keys for the two turns) — but production evidence refutes it as the cause here:

  • Pulled the router logs for the reporter's own /fm opus Codex-desktop session (client_session_id 01a049c4…, 2026-08-28): one stable session_key across every turn, turnloop pin lookup hit … user_forced on all of them. No drift.
  • Scanned a day of prod turnloop pin lookup hit lines: exactly one session with two distinct keys for one client_session_id, and it was a Claude Code sub-agent boundary (prompt text changes entirely between turns), not a router command.
  • Codex (CLI and desktop) goes over the Responses API, where input accumulates the first user message on every request — messages[0] never changes, so the key can't drift. Claude Code sends metadata.user_id, which pins the key independently of message text.

The theoretical gap remains for a hypothetical chat-completions client whose first-ever message is a router command and which does not resend history; that is not any current client. Left unfixed deliberately — changing key derivation moves every pin and prompt-cache slot, which is not justified by a failure mode no client exercises.

A /force-model pin is dropped for the turn when its provider isn't in
EnabledProviders, the model is excluded, or it can't carry an image. The
drop is correct — serving it would 401 — but it was entirely silent: no
log line, no marker. The turn fell through to the scorer and served another
model while the user was still looking at "force-model applied:
claude-opus-5" from the previous turn.

Reported from Codex desktop: a pin to an Anthropic model acked as applied,
then GCP logs showed an OpenAI model serving every subsequent turn.

Log the drop with its reason (the maxed-out guard ten lines below already
does this) and render a marker naming the pin that didn't apply. The marker
deliberately runs before the same-model suppression gate, which would
otherwise hide precisely the turns where the pin stopped applying.

No routing behavior changes; this only makes an existing silent fallback
observable.

Co-Authored-By: Weave Router <router@workweave.ai>
Signed-off-by: munir-weave <munir@workweave.ai>
@claude

claude Bot commented Aug 28, 2026

Copy link
Copy Markdown

Claude finished @munir-weave's task —— View job


  • Run git diff and identify added comment blocks
  • Flag overly long/verbose comment blocks
  • Post review suggestions

Posted 4 advisory suggestion comments on the PR review. All are non-blocking comment-length nits; no logic/correctness issues were in scope.

@workweave-bot workweave-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Advisory only — comment-length nits. Won't block merge.

Comment thread internal/proxy/service_session_pin_test.go Outdated
Comment thread internal/proxy/turnloop.go Outdated
Comment thread internal/proxy/turnloop.go Outdated
Comment thread internal/proxy/service.go Outdated
Co-Authored-By: Weave Router <router@workweave.ai>
Signed-off-by: munir-weave <munir@workweave.ai>
@claude

claude Bot commented Aug 28, 2026

Copy link
Copy Markdown

Claude finished @munir-weave's task —— View job


Claude finished @munir-weave's task —— View job


  • Run git diff and identify added comment blocks
  • Flag overly long/verbose comment blocks
  • Post review suggestions (or exit silently if nothing to flag)

All previously-flagged comment blocks have been trimmed per the prior review. Nothing in the current diff meets the flagging criteria — no new suggestions posted.

@greptile-apps

greptile-apps Bot commented Aug 28, 2026

Copy link
Copy Markdown

T-Rex T-Rex Logs

What T-Rex did

  • T-Rex ran the requested verification.
  • During the verification, it was observed that local artifact references were not uploaded.

T-Rex Ran code and verified through T-Rex

Reviews (1): Last reviewed commit: "Surface a force-model pin that could not..." | Re-trigger Greptile

@munir-weave
munir-weave merged commit 2057381 into main Aug 28, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants