On a CONTEXT_WINDOW_EXCEEDED failure (dsh core's canonical code for a 400 "maximum context length"), the plugin already switches if the code is in triggerCodes — verified with dsh-llm-fallbacks 0.4.2 against a fake provider. Three things are off, one of them undocumented:
commit() suppresses the from route for cooldownMs (escalating under half-open recovery). The route is healthy; the request was too big. The next short prompt goes to the fallback for no reason, and a half-open probe is spent.- Candidates are not filtered by context window, so the chain can fall over to a model that cannot fit the request either — which then fails and gets cooled down too.
- README does not say
triggerCodes accepts CONTEXT_WINDOW_EXCEEDED, or what happens to compaction when it does.
What I measured (Failover Bench: a fake OpenAI-compatible provider whose fb-s12-context-length model answers every request with a 400 "maximum context length"; dsh 0.1.2-rc.1 through the Python SDK; chain [fake/fb-ok]):
- default
triggerCodes: two attempts on the primary, then an error — the plugin never sees the code.
- with
CONTEXT_WINDOW_EXCEEDED added: one attempt on the primary, immediate switch, complete answer from the fallback in 2.6 s. Compaction was not attempted — this plugin's agent/request-error listener handled the rejection before compaction-basic got to it.
- the cooldown in (1) follows from
commit() calling states.suppress unconditionally; I could not observe it from outside, because each harness request runs in a fresh process.
Repro: python -m failoverbench run --system systems/dsh.yaml --profile full --only S12 in https://github.com/failoverbench/failoverbench (gateways/dsh/setup.sh writes the two harness homes and their settings).
Proposal (PR to follow): request-scoped switches (scope: 'request') that skip the suppress/recovery bookkeeping; a context-window candidate filter using the catalog's contextWindow when present, else llm.resolveModelInfo(provider, model).context.contextWindow (unknown → keep the candidate); README/CONCEPTS text.
Two questions for you: should CONTEXT_WINDOW_EXCEEDED be in the default triggerCodes? And should a context-window trigger give compaction the first try (switch only if the compacted retry still overflows), or switch immediately, as it does today?
On a
CONTEXT_WINDOW_EXCEEDEDfailure (dsh core's canonical code for a 400 "maximum context length"), the plugin already switches if the code is intriggerCodes— verified with dsh-llm-fallbacks 0.4.2 against a fake provider. Three things are off, one of them undocumented:commit()suppresses the from route forcooldownMs(escalating under half-open recovery). The route is healthy; the request was too big. The next short prompt goes to the fallback for no reason, and a half-open probe is spent.triggerCodesacceptsCONTEXT_WINDOW_EXCEEDED, or what happens to compaction when it does.What I measured (Failover Bench: a fake OpenAI-compatible provider whose
fb-s12-context-lengthmodel answers every request with a 400 "maximum context length"; dsh 0.1.2-rc.1 through the Python SDK; chain[fake/fb-ok]):triggerCodes: two attempts on the primary, then an error — the plugin never sees the code.CONTEXT_WINDOW_EXCEEDEDadded: one attempt on the primary, immediate switch, complete answer from the fallback in 2.6 s. Compaction was not attempted — this plugin'sagent/request-errorlistener handled the rejection before compaction-basic got to it.commit()callingstates.suppressunconditionally; I could not observe it from outside, because each harness request runs in a fresh process.Repro:
python -m failoverbench run --system systems/dsh.yaml --profile full --only S12in https://github.com/failoverbench/failoverbench (gateways/dsh/setup.shwrites the two harness homes and their settings).Proposal (PR to follow): request-scoped switches (
scope: 'request') that skip the suppress/recovery bookkeeping; a context-window candidate filter using the catalog'scontextWindowwhen present, elsellm.resolveModelInfo(provider, model).context.contextWindow(unknown → keep the candidate); README/CONCEPTS text.Two questions for you: should
CONTEXT_WINDOW_EXCEEDEDbe in the defaulttriggerCodes? And should a context-window trigger give compaction the first try (switch only if the compacted retry still overflows), or switch immediately, as it does today?