Description
When automatic model selection and smart routing are enabled, a conversation can become stuck on the simple tier.
After an earlier request selects a simple model and gets a cache hit, later requests that require a stronger tier may still use the same simple model. The cache-aware switching logic compares costs without distinguishing between upgrades and downgrades. Since upgrading is normally more expensive, the previous cached model may keep winning the comparison.
The screenshot below shows three turns using the same simple model. The final request asks for in-depth research and a long report, but the router still keeps it on simple.
Expected behavior
Cache cost should not prevent a judge-requested tier upgrade. It should only affect downgrade decisions.
Description
When automatic model selection and smart routing are enabled, a conversation can become stuck on the
simpletier.After an earlier request selects a
simplemodel and gets a cache hit, later requests that require a stronger tier may still use the samesimplemodel. The cache-aware switching logic compares costs without distinguishing between upgrades and downgrades. Since upgrading is normally more expensive, the previous cached model may keep winning the comparison.The screenshot below shows three turns using the same
simplemodel. The final request asks for in-depth research and a long report, but the router still keeps it onsimple.Expected behavior
Cache cost should not prevent a judge-requested tier upgrade. It should only affect downgrade decisions.