Skip to content

feat(gateway): a worker is sent only requests its catalog says it can serve - #207

Merged
V3RON merged 3 commits into
mainfrom
task/191
Oct 2, 2026
Merged

V3RON merged 3 commits into
mainfrom
task/191

Conversation

@V3RON

@V3RON V3RON commented Oct 2, 2026

Copy link
Copy Markdown
Contributor

Closes #191

The gateway now sends a worker only requests its catalog says it can serve. Before, it matched model names exactly, checked the model and runtime separately, and counted a download as available. So a name that works on a worker found no worker, and a worker that could not pair model and runtime got the request and refused it.

One matcher, routing/request-match.ts, decides this from the catalog. It finds the first model whose name or alias matches the request in any letter case, then checks the runtime against that model's paired runtimes. The can-serve stage, the warm-hit compare, and the forward all use it. The worker is sent its own name for the model and allowDownload: false. takes-requests and can-serve replace eligible in the registered policy. eligible stays only so the conformance tests can run the three legacy stages.

The docs now say that --device matches by any name and letter case, that model and runtime must pair, and that --allow-download starts nothing through a gateway. The "does not yet" sentences this change made false are gone.

Done when

  • Every test in the spec passes: case, alias, unpaired runtime, no runtime, allowDownload: true, and the e2e in fleet-coordinator.test.ts, request-match.test.ts, and e2e/gateway-fleet.test.ts. Each was checked to fail against main's routing.
  • First match in models wins: request-match.test.ts, both orders.
  • Legacy stages pass every routing.test.ts case unchanged.
  • pnpm check green: 2263 unit, 79 e2e.
  • git diff origin/main -- src/gateway/routing.test.ts shows only the import and the construction line.

Review

Spec review: 8 findings, 1 fixed. Code review: 12 findings, 4 fixed.

Rejected:

  • warm-then-free.test.ts changed without being listed — the registered policy changed, so the oracle comparison must run against the legacy stages it was written for.
  • docs/CLIENT.md edited without being listed — it said the gateway does not yet route by pairings or aliases, which this change made false.
  • worker-link.ts comment now names lease.maxTtlMs — config.get does read it; the old comment left it out.
  • Done when needs a run the diff cannot show — pnpm check was run and is green.
  • routing.test.ts "warm-then-free policy" cases run the legacy list, not the shipped one (raised twice) — the spec requires those bodies unchanged against the legacy stages; new tests cover the shipped policy.
  • The conformance test no longer checks the registered policy (raised twice) — that is its purpose under the spec: legacy stages against the legacy oracle.
  • eligible duplicates takes-requests/can-serve (raised twice) — the spec keeps it exported for the conformance test only, with a comment saying so.
  • A waiting request no worker can serve now queues instead of being refused by a worker — the spec puts failing it out of scope: "until then it waits, as on main".
  • Restoring allowDownload in routable() or the dispatcher leaves tests green — the shipped policy never reads it and the forward is hard-wired to false, which the allowDownload: true test proves.

Written by an agent.

V3RON added 3 commits October 2, 2026 08:22
… serve

Routing matches the model by name or alias in any letter case, requires the
catalog to pair it with the runtime, and forwards the worker's own name with
allowDownload: false. takes-requests and can-serve replace eligible in the
registered policy; eligible stays for the conformance tests.

Refs #191
@V3RON
V3RON merged commit 997691e into main Oct 2, 2026
14 checks passed
@V3RON
V3RON deleted the task/191 branch October 2, 2026 07:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A worker is sent only requests its catalog says it can serve

1 participant