--offload-config is parsed once in src/vllm/entrypoints/openai/server_main.cpp, AFTER the architecture resolution, so two of the server's own engine-construction paths build their EngineParams without it: the POOLING/embedding path (the if (pooling_model) block) and the transcription-only path. vllm-cli has no such flag at all.
This is not specific to the vllm_cpp residency extension that #1110 added. The MIRRORED uva/prefetch half is dropped on the same two paths and has been since before that key existed, so the residency key inherits a pre-existing gap rather than introducing one. It is recorded here because it was being tracked under #1122, which PR #1119 closes — a gap owned by a closed issue has no owner.
Consequence for a user: an embedding or transcription server started with --offload-config silently ignores it. Weights are placed as though the flag were absent, and nothing says so. The residency half of that is the 370 GiB case, where the difference is whether the process fits in host RAM at all.
Fixing the server half means moving the offload parse ahead of the architecture branch, which is ENG-WEIGHT-OFFLOAD's surface as much as ENG-RESIDENCY-CONFIG's. vllm-cli having no flag is a deliberate scope line for #1110 (that row adds no new flag) rather than a defect, but it belongs in the same record so a reader does not have to rediscover which of the three is which.
Both are documented in docs/USAGE.md beside the config form, and listed under ## Owed in .agents/specs/weight-residency-config.md.
Owning row: ENG-RESIDENCY-CONFIG.
--offload-configis parsed once insrc/vllm/entrypoints/openai/server_main.cpp, AFTER the architecture resolution, so two of the server's own engine-construction paths build theirEngineParamswithout it: the POOLING/embedding path (theif (pooling_model)block) and the transcription-only path.vllm-clihas no such flag at all.This is not specific to the
vllm_cppresidency extension that #1110 added. The MIRROREDuva/prefetchhalf is dropped on the same two paths and has been since before that key existed, so the residency key inherits a pre-existing gap rather than introducing one. It is recorded here because it was being tracked under #1122, which PR #1119 closes — a gap owned by a closed issue has no owner.Consequence for a user: an embedding or transcription server started with
--offload-configsilently ignores it. Weights are placed as though the flag were absent, and nothing says so. The residency half of that is the 370 GiB case, where the difference is whether the process fits in host RAM at all.Fixing the server half means moving the offload parse ahead of the architecture branch, which is
ENG-WEIGHT-OFFLOAD's surface as much asENG-RESIDENCY-CONFIG's.vllm-clihaving no flag is a deliberate scope line for #1110 (that row adds no new flag) rather than a defect, but it belongs in the same record so a reader does not have to rediscover which of the three is which.Both are documented in
docs/USAGE.mdbeside the config form, and listed under## Owedin.agents/specs/weight-residency-config.md.Owning row:
ENG-RESIDENCY-CONFIG.