Skip to content

KV-WARMUP-PROFILE's three upstream anchors all point at unrelated code at the current pin, and gguf_device_fit.h inherited two of them #1139

Description

@localai-bot

All three upstream anchors on the KV-WARMUP-PROFILE row in .agents/engine-matrix.md:112 point at unrelated code at the current parity pin 555967922. Verified by reading the pinned tree, not by inference:

Recorded anchor What is actually there at 555967922
vllm/v1/worker/gpu/model_runner.py:504 inside self.speculator.set_attn(...) for a DraftModelSpeculator (:500-508)
vllm/v1/worker/gpu/model_runner.py:647 a torch.zeros(...) argument inside a dummy_run=True construction (:643-652)
vllm/v1/worker/gpu_worker.py:430 a comment about max_split_size_mb inside load_model (:426-433)

The startup memory profile the row describes is real and is elsewhere:

  • GPUWorker.determine_available_memory, vllm/v1/worker/gpu_worker.py:451-495, which runs memory_profiling(...) (:491-494) around profile_run() (:495) and passes the weight bytes in as weights_memory=int(self.model_runner.model_memory_usage) (:493).
  • GPUModelRunner.profile_run, vllm/v1/worker/gpu/model_runner.py:682.
  • model_memory_usage itself is recorded AFTER the load finishes, vllm/v1/worker/gpu/model_runner.py:315.

That last point matters beyond the anchor: upstream measures the weight footprint only once the weights are resident, so it never asks whether they will fit. It is why ENG-EXPERT-STREAM's load-time GGUF fit refusal (#1123) has no upstream counterpart to mirror, and stating it from the real anchors is a stronger claim than the one the stale pair supported.

How this spread. include/vllm/model_executor/model_loader/gguf_device_fit.h and .agents/specs/expert-streaming.md both cited model_runner.py:504,647 for exactly this reason: they copied the pair from this row. Both are corrected in #1132 / #1136. This row is the source and still carries them, so the next reader inherits the same two anchors.

Why it is filed and not fixed in flow. The fix is one table cell in .agents/engine-matrix.md. PR #1119 (ENG-RESIDENCY-CONFIG, #1110) is in flight and bumps that file's ENGINE counter, and scripts/check-agent-record.py hardcodes the count, so two concurrent edits to that file are the record-lock hazard AGENTS.md names. The repairing session for #1136 was instructed not to touch either file for that reason. Owner: KV-WARMUP-PROFILE, whose planned spec is specs/warmup-memory-profiling.md.

Most likely cause: the anchors were correct at the previous e24d1b24 / 0.25.0 pin and were not reconciled when the pin advanced to 555967922. Worth checking whether other INVENTORIED rows carrying vllm/v1/worker/gpu/** anchors have the same problem, which is a bigger sweep than one cell.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions