feat(ENG-HF-MODEL-DOWNLOAD W3/W4): the downloader and the --model grammar, so --model org/repo fetches and serves (#1280)

CMakeLists.txt

@@ -2347,6 +2347,12 @@ if(VLLM_CPP_SERVER)
   # vendored cpp-httplib header api_server.cpp uses, so it shares that gate. The
   # cache half above stays unconditional because it opens no socket.
   target_sources(vllm PRIVATE src/vllm/transformers_utils/hf_hub.cpp)
+  # ENG-HF-MODEL-DOWNLOAD W3/W4 (#1280): the byte transport and the `--model`
+  # grammar. Both speak HTTP through the same vendored header, so both share
+  # api_server.cpp's gate. The resolver is what `server_main.cpp` calls, so a
+  # build without the server has no `--model` flag to resolve either.
+  target_sources(vllm PRIVATE src/vllm/transformers_utils/downloader.cpp)
+  target_sources(vllm PRIVATE src/vllm/transformers_utils/model_resolver.cpp)
   target_compile_definitions(vllm PUBLIC VLLM_CPP_SERVER)
   target_link_libraries(vllm PUBLIC Threads::Threads)
   # third_party/httplib/httplib.h is reached as <httplib/httplib.h> (third_party

docs/guides/hugging-face-access.md

@@ -1,7 +1,78 @@
 # Access Hugging Face checkpoints
 
-Pass a local directory or a `.gguf` file to `--model`. The CLI and server do
-not download a repository identifier yet.
+`--model` takes four forms, and leaves anything else alone. The two local forms
+are probed FIRST, so a network call can never shadow a path that exists on
+disk.
+
+| `--model` value | What happens |
+|---|---|
+| a directory | Opened as it always was. No network |
+| a `.gguf` file | Opened as it always was. No network |
+| `org/repo` | The checkpoint is fetched into the Hugging Face cache and the snapshot directory is served |
+| `org/repo:Q4_K_M` | ONE GGUF file for that quantization is fetched and served |
+| anything else | The error it produced before, unchanged |
+
+`org/repo` mirrors vLLM, which is the only upstream that defines it.
+`org/repo:QUANT` is llama.cpp's form and vLLM does not implement it, so it is a
+tracked divergence rather than a silent one.
+
+Two flags go with them, spelled as vLLM spells them. There is deliberately no
+inline `org/repo@revision` syntax, because vLLM does not have one.
+
+| Flag | Meaning |
+|---|---|
+| `--revision <ref>` | A branch, a tag, or a 40 character commit. Applies to both hub forms |
+| `--download-dir <path>` | The directory that holds the `models--org--repo` folders. Overrides the cache root below |
+
+This is the SHAPE of a run on a machine that holds no checkpoint. The repository
+name is an illustration: no fetch from the live hub has been gated in this tree
+yet, because this build speaks no transport layer security and the hermetic
+suites run against an in-process fake hub. The online gate is owed by W5 of
+[the Hugging Face download specification](../../.agents/specs/hf-model-download.md).
+
+```sh
+vllm-server --model Qwen/Qwen3-0.6B --port 8000
+curl localhost:8000/v1/completions \
+  -H 'Content-Type: application/json' \
+  -d '{"model":"Qwen/Qwen3-0.6B","prompt":"hello","max_tokens":16}'
+```
+
+The name a client puts in `"model"` is the name you typed, not the commit
+directory the cache happens to hold.
+
+**`vllm-server` is the only entry point that resolves a repository identifier.**
+`vllm-cli` and the C ABI's `vllm_model_load` still take a local path, because an
+example is an application binary interface client only and this work adds no ABI
+function.
+
+## How a fetch runs
+
+The fetch is TWO PHASE. The configuration JSON, the tokenizer and the shard
+index come first, and the weights follow, so a repository that is not a model
+fails after a few hundred kilobytes rather than after sixty gigabytes. It is
+also INDEX DRIVEN: when the repository ships a `model.safetensors.index.json`,
+the exact file names in its `weight_map` are fetched, so a published
+checkpoint's duplicate-format `original/` directory is never requested. Where
+there is no index, `*.safetensors` is preferred over `*.bin` and only the first
+of the two that matches is fetched.
+
+A transfer resumes. A partial file is kept as `{blob}.incomplete` and the next
+run asks for `Range: bytes=N-`. **A server that answers that request with `200`
+and the whole file is refused, not appended to**: appending a whole body to a
+partial file writes the first N bytes twice, and a corrupt weight still emits
+tokens, so nothing downstream would notice. Delete the named `.incomplete` file
+and run again to start over. `Ctrl-C` cancels and keeps the partial file.
+
+Nothing takes the name of a finished file until it has proven itself from its
+own bytes: a safetensors satisfies `8 + header_len + max(data_offsets[1]) ==
+file_size`, and a GGUF is opened by this project's own reader, which validates
+every tensor span against the real file size. A `Content-Length` that matches is
+not accepted as proof on its own.
+
+`--verbose` prints one line per file, and one lock is taken per repository so
+two servers started at once against one cache do not write one blob twice.
+
+## Environment
 
 The library reads these Hugging Face environment variables when it resolves a
 cache entry:
@@ -14,7 +85,9 @@ cache entry:
 | `HF_HUB_OFFLINE` | Resolve files from the cache without opening a network connection |
 | `HF_HUB_CACHE`, `HUGGINGFACE_HUB_CACHE`, `HF_HOME`, `XDG_CACHE_HOME`, `HOME` | Cache root, in priority order; `HF_HOME` contributes `$HF_HOME/hub` |
 
-The resolver reads the standard Hugging Face cache layout:
+## Cache layout
+
+The resolver reads and writes the standard Hugging Face cache layout:
 
 ```text
 {hub}/models--org--repo/
@@ -24,19 +97,38 @@ The resolver reads the standard Hugging Face cache layout:
 ```
 
 If the repository has multiple cached snapshots, the resolver selects the most
-recently written snapshot.
+recently written snapshot. A cache another tool already populated is read rather
+than re-downloaded.
+
+Where the file system holds no symbolic link, which is the case for a CIFS mount
+and can be the case for a `/cache` container volume, a snapshot entry becomes a
+real file, and the switch is logged one time for each cache directory it happens
+in.
+
+A cached blob is named by the object identifier the listing carried, and by the
+commit plus the repository-relative path when it carried none. `--verbose` says
+which of the two the run used.
+
+A repository listing is refused, rather than partly used, when it fails either
+of two integrity checks. An object identifier given to two entries that disagree
+on size is refused, because no content hash names two sizes. An identifier whose
+characters are all one repeated character is refused, because no content hash
+produces one, and an unauthenticated listing for a gated repository is the shape
+that produces it. An entry that reports no size can never own an identifier, so
+a mirror named by `HF_ENDPOINT` cannot switch the check off by omitting one
+field.
 
 ## Current limitations
 
-- The CLI and server do not fetch checkpoints. Setting `HF_TOKEN` does not
-  change a server request until repository downloads reach that entry point.
+- This build speaks plain hypertext transfer protocol only, so an `https`
+  endpoint is refused with a message naming the build options that would add
+  transport layer security rather than with a connection error. Point
+  `HF_ENDPOINT` at an `http` mirror until that lands.
+- `vllm-cli` and the C ABI do not resolve a repository identifier. Both still
+  take a local path.
 - The DFlash draft resolver reads `$HOME/.cache/huggingface/hub`. It does not
   honor `HF_HOME`.
-- Cache-writing support has no public caller yet.
-
-The cache writer supports filesystems without symbolic links, including CIFS
-mounts and some container volumes. On such filesystems it writes a regular file
-for each snapshot entry.
 
-See the [Hugging Face download specification](../../.agents/specs/hf-model-download.md)
-for implementation state, integrity checks, and remaining work.
+All three are recorded under `## Owed` in
+[the Hugging Face download specification](../../.agents/specs/hf-model-download.md),
+along with the integrity checks and the remaining work.

docs/reference/server.md

@@ -140,10 +140,12 @@ a stop token early.
 
 | Flag | Default | Meaning |
 |---|---|---|
-| `--model <dir>` | Required except when `--speech-model` selects a speech/music-only server | Model directory (safetensors or `.gguf`) |
+| `--model <dir\|file.gguf\|org/repo\|org/repo:QUANT>` | Required except when `--speech-model` selects a speech/music-only server | A local directory or `.gguf` file, opened as before, or a Hugging Face repository, which is fetched into the cache. The local forms are probed first, so a network call can never shadow a path on disk. See [Access Hugging Face checkpoints](../guides/hugging-face-access.md) |
+| `--revision <ref>` | repository default branch | A branch, a tag, or a 40 character commit for a `--model org/repo`. vLLM's own flag, and there is no inline `org/repo@rev` syntax |
+| `--download-dir <path>` | the resolved Hugging Face cache root | The directory that holds the `models--org--repo` folders. vLLM's own flag |
 | `--host H` | `0.0.0.0` | Bind host |
 | `--port P` | `8000` | Bind port |
-| `--served-model-name N` | model dir basename | Model id in `/v1/models` and responses |
+| `--served-model-name N` | model dir basename, or the `org/repo` you typed | Model id in `/v1/models` and responses |
 | `--tokenizer-config F` | `<dir>/tokenizer_config.json` | Chat template / tokenizer config |
 | `--block-size N` | `32` | KV block size. **Must be a multiple of 16**, the attention backends' `get_kv_cache_shape` refuses anything else, and the server now rejects it at startup rather than throwing during engine init |
 | `--num-blocks N` | `0` (auto, resolves to `256`) | KV block count, and vLLM's `num_gpu_blocks_override`. It wins over every other sizing knob. `0` means auto, which uses `--kv-cache-memory` when that is set and otherwise falls back to `256` blocks |

include/vllm/transformers_utils/downloader.h

@@ -0,0 +1,163 @@
+// vllm.cpp original: byte transport for a HuggingFace file. Probe, resume,
+// structural proof, cross-process lock and progress.
+//
+// vLLM reaches the bytes through `huggingface_hub`, so the wire behavior has no
+// vLLM source to mirror and the structural reference is the secondary oracle
+// llama.cpp at stock tag `b10451` (commit
+// `10bf611e533d81f739128304991c5e133c6aebd8`):
+//   - size, entity tag and range probe, `common/download.cpp:321-349`
+//   - range resume, `common/download.cpp:222-235`
+// The cross-process lock mirrors vLLM `weight_utils.py:506` at pin
+// `5559679229`, which takes one lock per model before it downloads.
+//
+// ONE BEHAVIOR DELIBERATELY DIVERGES FROM llama.cpp, and it is the reason this
+// file is not a port of that one. See `HubDownloadFile`.
+//
+// ENG-HF-MODEL-DOWNLOAD W3, issue #1280.
+#pragma once
+
+#include <cstdint>
+#include <filesystem>
+#include <optional>
+#include <string>
+
+#include "vllm/transformers_utils/hf_hub.h"
+
+namespace vllm {
+namespace transformers_utils {
+
+// What a `HEAD` probe learned about one remote file. Mirrors llama.cpp
+// `common/download.cpp:321-349 @ b10451`, which asks the same three questions.
+struct HfRemoteFile {
+  // `Content-Length`, and EMPTY when the answer carried none. It is an optional
+  // for the reason `HfFile::size` is one: a zero-byte file and an unanswered
+  // size are different facts, and `0` spells both. Every caller that sizes a
+  // byte range, a resume offset or a completeness check from it has to be able
+  // to tell them apart.
+  std::optional<uint64_t> size;
+  // The entity tag with its quotes and any `W/` prefix removed, or empty.
+  std::string etag;
+  // `Accept-Ranges: bytes`. A server that does not say so is never asked to
+  // resume, because a range it ignores is the failure mode below.
+  bool accepts_ranges = false;
+};
+
+// `HEAD {url}`, following a redirect to the content delivery network. The
+// bearer token is sent to the host the caller named and to NO redirect target,
+// for the reason `hf_hub.cpp`'s client does not set `set_follow_location`.
+//
+// Throws std::runtime_error on a refusal, and refuses before opening a socket
+// when `opts.offline` is set.
+HfRemoteFile HubProbeFile(const std::string& url, const HfHubOptions& opts);
+
+// The three file shapes this row can prove complete STRUCTURALLY, which is the
+// only proof it accepts. An opaque remote field is not one: a hub that is not
+// answering the truth about a repository can answer a length field too.
+enum class HfFileShape {
+  // Nothing about the interior is known. Only the byte count is checked.
+  kOpaque,
+  // `8 + header_len + max(data_offsets[1]) == file_size`.
+  kSafetensors,
+  // Magic, version, tensor count, and every tensor span inside the data end.
+  kGguf,
+};
+
+// The shape a repository-relative path declares by its extension.
+HfFileShape HfShapeForPath(const std::string& path);
+
+// Prove `file` complete for `shape`. Throws std::runtime_error naming the file
+// and the part that did not add up. `kOpaque` returns without reading a byte.
+void HfVerifyFileShape(const std::filesystem::path& file, HfFileShape shape);
+
+struct HfDownloadOptions {
+  HfHubOptions hub;
+  // Progress to standard error. The server spells this `--verbose`.
+  bool verbose = false;
+};
+
+// What one `HubDownloadFile` call did. `bytes_written` counts what THIS call
+// transferred, so a resumed transfer reports the tail and a cache hit reports
+// zero. That distinction is the only honest way for a test to state either.
+struct HfDownloadResult {
+  uint64_t bytes_written = 0;
+  uint64_t file_size = 0;
+  bool resumed = false;
+  bool already_present = false;
+  std::string etag;
+};
+
+// Fetch `url` into `dest`.
+//
+// The bytes land in `{dest}.incomplete` and the file is renamed to `dest` only
+// after `HfVerifyFileShape` passes, so a partial or malformed transfer never
+// occupies the name a later run reads as a cache hit.
+//
+// An existing `{dest}.incomplete` is RESUMED with `Range: bytes=N-`.
+//
+// THE DIVERGENCE. llama.cpp warns and continues when a range request is
+// answered `200` instead of `206` (`common/download.cpp:222-235 @ b10451`,
+// "server did not respond with 206 ... restarting download"; the fork's
+// variant logs and keeps the handle). We THROW. A `200` carries the WHOLE body,
+// and appending a whole body onto a partial file produces a file whose bytes
+// are the first N twice. That file has the wrong length, so the structural
+// check catches it here, but the same shape on a format with no structural
+// proof is a silently corrupt weight, and a token gate cannot see a corrupt
+// weight: it still emits tokens. A refusal that names the cause costs one
+// re-run. A corrupt shard costs a debugging session that starts at the model.
+//
+// `expected_size` is the size the TREE LISTING reported, and it is an optional
+// because the listing may report none. It is checked against the transferred
+// byte count when it has a value and is not invented when it does not.
+//
+// Cancels on `SIGINT` once `HfInstallDownloadInterruptHandler` has been called:
+// the partial file is left in place, so the next run resumes rather than
+// starting again.
+//
+// Throws std::runtime_error on any refusal, and opens no socket under
+// `opts.hub.offline`.
+HfDownloadResult HubDownloadFile(const std::string& url,
+                                 const std::filesystem::path& dest,
+                                 const std::optional<uint64_t>& expected_size,
+                                 HfFileShape shape,
+                                 const HfDownloadOptions& opts);
+
+// One lock per REPOSITORY, held across processes, mirroring vLLM
+// `weight_utils.py:506`. Two `vllm-server` processes started at once against
+// one cache must not write one blob twice.
+//
+// The lock file sits BESIDE the repository directory rather than inside it, so
+// it is never mistaken for a snapshot entry and never has to be filtered out of
+// the cache walk. An empty `repo_path` takes no lock, because a host with no
+// cache directory has nothing to serialize on.
+class HfRepoLock {
+ public:
+  explicit HfRepoLock(const std::filesystem::path& repo_path);
+  ~HfRepoLock();
+  HfRepoLock(const HfRepoLock&) = delete;
+  HfRepoLock& operator=(const HfRepoLock&) = delete;
+
+  // True when this object holds a file lock. False when the host has no cache
+  // directory, or when the platform has no advisory lock: the download still
+  // runs, because refusing to fetch a model because a lock could not be taken
+  // is worse than the race the lock prevents.
+  bool held() const { return fd_ >= 0; }
+  const std::filesystem::path& path() const { return path_; }
+
+ private:
+  int fd_ = -1;
+  std::filesystem::path path_;
+};
+
+// Install the `SIGINT` handler that cancels an in-flight transfer. Idempotent.
+// The previous handler is remembered and re-raised, so `Ctrl-C` still ends the
+// process rather than only ending the download.
+void HfInstallDownloadInterruptHandler();
+
+// True once `SIGINT` arrived after the handler was installed.
+bool HfDownloadInterrupted();
+
+// Clear the flag. For tests, and for a caller that handled the cancellation.
+void HfResetDownloadInterrupt();
+
+}  // namespace transformers_utils
+}  // namespace vllm

include/vllm/transformers_utils/hf_hub.h

@@ -153,6 +153,34 @@ std::string HubResolveCommitCached(const std::string& repo_id,
                                    const std::string& revision,
                                    const HfHubOptions& opts);
 
+// A parsed hypertext address, and the two checks every call in this file and
+// in `downloader.h` runs before it opens a socket. They are declared HERE so
+// the byte transport speaks ONE address grammar with the protocol half rather
+// than growing a second one that drifts.
+//
+// Mirrors llama.cpp `common/http.h:33-98 @ b10451`, narrowed to what a hub
+// address needs: no user information, because a hub endpoint carries none.
+struct HfParsedUrl {
+  std::string scheme;
+  std::string host;
+  int port = 0;
+  std::string path;  // always begins with '/'
+};
+
+// Throws std::runtime_error naming `url` on a missing scheme, an unterminated
+// bracketed IPv6 authority, an unsupported scheme, or an empty host.
+HfParsedUrl HfParseUrl(const std::string& url);
+
+// `[host]` for an IPv6 literal, `host` otherwise. What httplib's scheme-host-port
+// constructor needs.
+std::string HfFormatHost(const std::string& host);
+
+// Throw when `url` is `https` and this build carries no transport layer
+// security, with a message that NAMES the three build options. A build where
+// the option resolved OFF otherwise fails with a connection error that reads
+// like a network fault.
+void HfRefuseHttpsWithoutTls(const std::string& url);
+
 // True when `repo_id` has the shape the hub accepts: base characters
 // `[A-Za-z0-9_]`, the special characters `/.-` only between base characters,
 // and exactly one '/'. Mirrors llama.cpp `common/hf-cache.cpp:121-142`.