Stop reporting a not-ready media model as 405 Method Not Allowed#4274
Open
stisiTT wants to merge 1 commit into
Open
Stop reporting a not-ready media model as 405 Method Not Allowed#4274stisiTT wants to merge 1 commit into
stisiTT wants to merge 1 commit into
Conversation
stisiTT
requested review from
a team,
ddjukicTT,
dmadicTT,
fivanovicTT,
idjuricTT,
knovokmetTT,
ljovanovicTT,
lmuravljovTT,
nanicicTT,
visnjakrsmanovicTT,
vpetrovicTT and
ztorlakTT
as code owners
The scheduler raised HTTP 405 ("Method Not Allowed") while the model was
still warming up, which is semantically wrong and confused health probes:
a shield liveness probe polling /tt-liveness received 405 and treated the
server as broken rather than warming up.
Use 503 ("Service Unavailable") for every not-ready state so probes,
the cloud console, and k8s all interpret it as "retry later." Applies
the same fix to the inference endpoints (chat, llm, audio, video,
fine_tuning) that independently re-raised 405 on the not-ready path.
Contributor
Author
tt-shield validation runsDispatched on-dispatch Targeted runs
Blackhole galaxy (
|
| Model | Run |
|---|---|
| whisper-large-v3 | link |
| Wan2.2-T2V-A14B | link |
| Wan2.2-I2V-A14B | link |
| stable-diffusion-xl-base-1.0 | link |
| mochi-1-preview | link |
Wormhole galaxy (6u / galaxy)
| Model | Run |
|---|---|
| whisper-large-v3 | link |
| FLUX.1-dev | link |
| Wan2.2-T2V-A14B | link |
| stable-diffusion-xl-base-1.0 | link |
| mochi-1-preview | link |
Models excluded from galaxy coverage (no galaxy spec): speecht5_tts, FLUX.1-schnell, Z-Image-Turbo.
stisiTT
changed the title
This was referenced Jun 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this fixes
When a media model isn't ready yet (still warming up), the server replied to requests with HTTP 405 Method Not Allowed. That status is the wrong kind of signal: 405 means "you're calling this endpoint the wrong way" — a client/method/contract error. It says nothing about the model, and it points whoever sees it at the wrong problem.
That mislabeling had real consequences. Our CI health probe hit the endpoint, saw 405, and concluded the endpoint itself was broken — so it gave up instead of waiting for warmup. A human reading the same 405 would reasonably go check routing, methods, or the URL — none of which were actually wrong.
The fix makes every not-ready path return 503 Service Unavailable, which truthfully describes the situation: the endpoint is correct, the service is just temporarily not ready. Callers (CI, the cloud console, Kubernetes) already treat 503 as "retry shortly," so nothing on their side changes.
This is purely about what the endpoint reports. It does not change which endpoint anything hits, the routes, the methods, or the paths.
What changed
/tt-livenessdocstring and two tests to match.Tested
tests/test_scheduler.py— 23 passedtests/test_tt_maintenance_api.py— 12 passedOut of scope (separate follow-up)
Two related-but-distinct items are not in this PR: (1) which endpoint consumers should probe (
/healthvs/tt-liveness), and (2) making the server actually detect a hung (not just not-ready) model and report it unhealthy. Those are tracked separately.