You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Qwen3.8-2.4T-A95B UD-Q1_0 serves on one GB10 today: the borrowed-mmap tier makes it
fit (ENG-EXPERT-STREAM, #912), --offload-config's vllm_cpp key configures it
(#1119), and --device cuda is refused at load with the footprint arithmetic instead of
dying mid-stream (#1123, #1132). None of that is reachable from the docs as a recipe.
The checkpoint is not pinned anywhere a user reads. AGENTS.md requires every ported
model to document the checkpoints it was built and gated against in docs/USAGE.md —
file name, size, exact repo and revision, sha256 for a quantized artifact. Revision 567d3e6ac26c5474b18311e619c04350fb9a5556 appears only inside .agents/issue-index.md
row #1123 and .agents/specs/expert-streaming.md. docs/USAGE.md names the checkpoint
three times and pins it zero times, so nobody outside the agent record can reproduce a
single number this project publishes about it.
Nor is there an end-to-end recipe.docs/USAGE.md's "Streaming routed experts from
disk (capacity mode)" is a good reference for the mechanism — schema, precedence, where
the config form does not reach, the statistics line, the CUDA refusal — and it is only
that. It does not say which file to point --model at, what the load costs, what decode
costs, or where the ceiling is. The four sibling per-model recipes in the same file
(Muse Glimmer 30B, Nemotron-3.5-Lightning-30B, MiniMax-H3, MiniMax-Music3) all do.
Two defects in that section, found while reading it.
Both example invocations name -00001-of-00008.gguf. The checkpoint has ten
shards: UD-Q1_0/Qwen3.8-2.4T-A95B-UD-Q1_0-00001-of-00010.gguf through -00010-of-00010.gguf, 397,256,393,248 bytes total, confirmed against the HF API at
the pinned revision and consistent with the 1702-record census in Qwen3.8-2.4T on --device cuda: the model loads, then the first forward dies on cudaMalloc: out of memory #1123. Copying the
published line gets a file-not-found.
Line 3848 writes the server as ./build/vllm-server. examples/ is added with add_subdirectory and nothing overrides RUNTIME_OUTPUT_DIRECTORY, so the binary is
at build/examples/vllm-server — which is what the same file writes at lines 1401 and
2634, and what .agents/specs/expert-streaming.md records as the working invocation.
Line 2354 carries the same defect for Muse Glimmer.
Not a gap in the mechanism reference. The fix is a recipe section beside its four
siblings that links to the streaming section rather than restating it, so each fact keeps
one home.
Qwen3.8-2.4T-A95B UD-Q1_0serves on one GB10 today: the borrowed-mmap tier makes itfit (
ENG-EXPERT-STREAM, #912),--offload-config'svllm_cppkey configures it(#1119), and
--device cudais refused at load with the footprint arithmetic instead ofdying mid-stream (#1123, #1132). None of that is reachable from the docs as a recipe.
The checkpoint is not pinned anywhere a user reads. AGENTS.md requires every ported
model to document the checkpoints it was built and gated against in
docs/USAGE.md—file name, size, exact repo and revision, sha256 for a quantized artifact. Revision
567d3e6ac26c5474b18311e619c04350fb9a5556appears only inside.agents/issue-index.mdrow #1123 and
.agents/specs/expert-streaming.md.docs/USAGE.mdnames the checkpointthree times and pins it zero times, so nobody outside the agent record can reproduce a
single number this project publishes about it.
Nor is there an end-to-end recipe.
docs/USAGE.md's "Streaming routed experts fromdisk (capacity mode)" is a good reference for the mechanism — schema, precedence, where
the config form does not reach, the statistics line, the CUDA refusal — and it is only
that. It does not say which file to point
--modelat, what the load costs, what decodecosts, or where the ceiling is. The four sibling per-model recipes in the same file
(Muse Glimmer 30B, Nemotron-3.5-Lightning-30B, MiniMax-H3, MiniMax-Music3) all do.
Two defects in that section, found while reading it.
-00001-of-00008.gguf. The checkpoint has tenshards:
UD-Q1_0/Qwen3.8-2.4T-A95B-UD-Q1_0-00001-of-00010.ggufthrough-00010-of-00010.gguf, 397,256,393,248 bytes total, confirmed against the HF API atthe pinned revision and consistent with the 1702-record census in Qwen3.8-2.4T on --device cuda: the model loads, then the first forward dies on cudaMalloc: out of memory #1123. Copying the
published line gets a file-not-found.
./build/vllm-server.examples/is added withadd_subdirectoryand nothing overridesRUNTIME_OUTPUT_DIRECTORY, so the binary isat
build/examples/vllm-server— which is what the same file writes at lines 1401 and2634, and what
.agents/specs/expert-streaming.mdrecords as the working invocation.Line 2354 carries the same defect for Muse Glimmer.
Not a gap in the mechanism reference. The fix is a recipe section beside its four
siblings that links to the streaming section rather than restating it, so each fact keeps
one home.