Skip to content

docs/ENVIRONMENT.md still says Metal does not advertise host-addressable device memory, and ROCm is unlisted, since cffe59b02 #1502

Description

@localai-bot

docs/ENVIRONMENT.md says Metal does not advertise host-addressable device memory. Since cffe59b02 it does.

VT_ADOPT_DEVICE_BYTES's row in docs/ENVIRONMENT.md describes its reach as:

only acts where the backend advertises host-addressable device memory — Vulkan today

and closes with:

No effect on CUDA/CPU/Metal, whose backends do not advertise the property

Both halves are now wrong, and ROCm is not mentioned at all.

cffe59b02 (#1477, Fixes #844) changed ReferenceTierEligible to gate on
Backend::DeviceMemoryIsHostAddressable() rather than UnifiedMemory(), and
added truthful overrides so no backend silently lost the reference tier:

  • src/vt/metal/metal_backend.mm:123DeviceMemoryIsHostAddressable() returns
    the Metal context's unified-memory answer.
  • src/vt/rocm/rocm_backend.hip:371 — returns unified_memory_.

src/vllm/model_executor/models/qwen3_5_weights.cpp:345,365 gates the
VT_ADOPT_DEVICE_BYTES lever on exactly that predicate. So Metal, and ROCm on
an integrated part, now answer it true
and the lever acts there.

What is actually wrong, precisely

  1. "Vulkan today" undercounts the set.
  2. "No effect on … Metal" is false.
  3. ROCm is absent from a sentence that enumerates the backends.

Whether the lever's behaviour on those backends is desirable is a separate
question this issue does not answer — #1477's review established the widening
is sound for the reference tier (every Vulkan allocation is HOST_VISIBLE | HOST_COHERENT and persistently mapped, guaranteed by a VT_CHECK), but nobody
has measured VT_ADOPT_DEVICE_BYTES on Metal or on integrated ROCm. The
measured numbers quoted in that row are GB10 and Vulkan only.

So the correction is not merely "add two backend names". It should say which
backends the lever is MEASURED on and which merely now satisfy the predicate,
because those are different claims and the row currently reads as if the
measurement covers the reach.

Provenance, and why it is not a merge artifact

The line is byte-identical at #1477's merge base and on origin/main before
the merge, and it was present at the reviewed head that passed CI. So this is a
consequence of the change landing, not of the merge that landed it — the review
and every gate looked at the code and nothing tied that code to this document.
No checker relates a backend predicate to a prose sentence in
docs/ENVIRONMENT.md, and none can be built cheaply, which is why this is a
found-by-reading defect rather than a caught-by-gate one.

Found while verifying main after merging #1477.

Owner

Owned by row VT-REFTIER-HOST-ADDRESSABLE, whose spec is
.agents/specs/vt-reference-tier-host-addressable.md. Not fixed in the landing
flow because it was found after the merge; recording it here rather than leaving
the document to be believed.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions