Surface a force-model pin that could not be served - #1111
Merged
Conversation
A /force-model pin is dropped for the turn when its provider isn't in EnabledProviders, the model is excluded, or it can't carry an image. The drop is correct — serving it would 401 — but it was entirely silent: no log line, no marker. The turn fell through to the scorer and served another model while the user was still looking at "force-model applied: claude-opus-5" from the previous turn. Reported from Codex desktop: a pin to an Anthropic model acked as applied, then GCP logs showed an OpenAI model serving every subsequent turn. Log the drop with its reason (the maxed-out guard ten lines below already does this) and render a marker naming the pin that didn't apply. The marker deliberately runs before the same-model suppression gate, which would otherwise hide precisely the turns where the pin stopped applying. No routing behavior changes; this only makes an existing silent fallback observable. Co-Authored-By: Weave Router <router@workweave.ai> Signed-off-by: munir-weave <munir@workweave.ai>
|
Claude finished @munir-weave's task —— View job
Posted 4 advisory suggestion comments on the PR review. All are non-blocking comment-length nits; no logic/correctness issues were in scope. |
workweave-bot
left a comment
Collaborator
There was a problem hiding this comment.
Advisory only — comment-length nits. Won't block merge.
Co-Authored-By: Weave Router <router@workweave.ai> Signed-off-by: munir-weave <munir@workweave.ai>
|
Claude finished @munir-weave's task —— View job Claude finished @munir-weave's task —— View job
All previously-flagged comment blocks have been trimmed per the prior review. Nothing in the current diff meets the flagging criteria — no new suggestions posted. |
What T-Rex did
Reviews (1): Last reviewed commit: "Surface a force-model pin that could not..." | Re-trigger Greptile |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
A user's
/force-modelpin is dropped for the turn when its provider isn't inEnabledProviders, the model is excluded, or it can't carry an image (turnloop.go). Dropping it is correct — dispatching to a provider the request can't authenticate to would 401. But the drop was completely silent: no log line, no marker. The turn fell through to the scorer and served a different model while the user was still looking atforce-model applied: claude-opus-5 (anthropic)from the previous turn.Reported from Codex desktop: a coworker pinned an Anthropic model, got the success ack, then checked GCP/router logs and found an OpenAI model serving every subsequent turn.
Two things made this hard to see:
forcedModelBindingvalidates the org allowlist, installation exclusions, gateway bindings, and policy-excluded providers — but not the enabled-provider set the next turn applies. So validation and enforcement disagree, and the ack is written a turn before the conflict surfaces.pinFound = falsereset discards the pin with no trace. The maxed-out guard immediately below it does log its drop ("Session pin maxed out on previous turn"), which is the precedent this follows.Change
drop_reason(provider_not_enabled/excluded/not_image_capable).turnLoopResultand render a routing marker naming the pin that didn't apply:✦ **Weave Router** → gpt-5.5 · your force-model pin could not be served this turn (claude-opus-5).The marker branch runs before the same-model suppression gate. That gate compares against the prior served model, so it would otherwise hide exactly the turns where the pin quietly stopped applying. Suggestion mode and empty decisions still suppress as before.
No routing behavior changes — this only makes an existing silent fallback observable.
Testing
TestService_SessionPin_ForcedPinDropped_SurfacesInMarker— end-to-end: an Anthropic-bound forced pin with only OpenAI wired falls through to the scorer, servesgpt-5.5, and surfaces the dropped pin.routingMarkerForcases: shown even when the served model is unchanged; suppressed in suggestion mode.go test ./...passes with it.Session-key derivation — investigated, not a live bug
A candidate second cause was
DeriveSessionKeyhashingFirstUserMessageText(): if/fm …is the first user message, the command is stripped from the body, so the next turn would derive a different session key and miss the pin. I wrote the unit test and it does reproduce (two different 16-byte keys for the two turns) — but production evidence refutes it as the cause here:/fm opusCodex-desktop session (client_session_id 01a049c4…, 2026-08-28): one stablesession_keyacross every turn,turnloop pin lookup hit … user_forcedon all of them. No drift.turnloop pin lookup hitlines: exactly one session with two distinct keys for oneclient_session_id, and it was a Claude Code sub-agent boundary (prompt text changes entirely between turns), not a router command.inputaccumulates the first user message on every request —messages[0]never changes, so the key can't drift. Claude Code sendsmetadata.user_id, which pins the key independently of message text.The theoretical gap remains for a hypothetical chat-completions client whose first-ever message is a router command and which does not resend history; that is not any current client. Left unfixed deliberately — changing key derivation moves every pin and prompt-cache slot, which is not justified by a failure mode no client exercises.