Conversation
… serve Routing matches the model by name or alias in any letter case, requires the catalog to pair it with the runtime, and forwards the worker's own name with allowDownload: false. takes-requests and can-serve replace eligible in the registered policy; eligible stays for the conformance tests. Refs #191
…ed alias keys Refs #191
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #191
The gateway now sends a worker only requests its catalog says it can serve. Before, it matched model names exactly, checked the model and runtime separately, and counted a download as available. So a name that works on a worker found no worker, and a worker that could not pair model and runtime got the request and refused it.
One matcher,
routing/request-match.ts, decides this from the catalog. It finds the first model whose name or alias matches the request in any letter case, then checks the runtime against that model's paired runtimes. Thecan-servestage, the warm-hit compare, and the forward all use it. The worker is sent its own name for the model andallowDownload: false.takes-requestsandcan-servereplaceeligiblein the registered policy.eligiblestays only so the conformance tests can run the three legacy stages.The docs now say that
--devicematches by any name and letter case, that model and runtime must pair, and that--allow-downloadstarts nothing through a gateway. The "does not yet" sentences this change made false are gone.Done when
allowDownload: true, and the e2e infleet-coordinator.test.ts,request-match.test.ts, ande2e/gateway-fleet.test.ts. Each was checked to fail againstmain's routing.modelswins:request-match.test.ts, both orders.routing.test.tscase unchanged.pnpm checkgreen: 2263 unit, 79 e2e.git diff origin/main -- src/gateway/routing.test.tsshows only the import and the construction line.Review
Spec review: 8 findings, 1 fixed. Code review: 12 findings, 4 fixed.
Rejected:
warm-then-free.test.tschanged without being listed — the registered policy changed, so the oracle comparison must run against the legacy stages it was written for.docs/CLIENT.mdedited without being listed — it said the gateway does not yet route by pairings or aliases, which this change made false.worker-link.tscomment now nameslease.maxTtlMs—config.getdoes read it; the old comment left it out.pnpm checkwas run and is green.routing.test.ts"warm-then-free policy" cases run the legacy list, not the shipped one (raised twice) — the spec requires those bodies unchanged against the legacy stages; new tests cover the shipped policy.eligibleduplicatestakes-requests/can-serve(raised twice) — the spec keeps it exported for the conformance test only, with a comment saying so.main".allowDownloadinroutable()or the dispatcher leaves tests green — the shipped policy never reads it and the forward is hard-wired tofalse, which theallowDownload: truetest proves.Written by an agent.