Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
.git
.gitignore
.pytest_cache
.ruff_cache
__pycache__
*.py[cod]
build
dist
*.egg-info
vllm_musa.egg-info
.coverage
htmlcov

# setup.py clones and pins third_party sources inside the image. Copying a local
# checkout makes the build context large and can hide drift from the pinned refs.
third_party/vllm
third_party/flashinfer

# Local/editor artifacts.
.claude
.humanize
assets
15 changes: 12 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ The plugin leverages the following key components:

| vLLM Version | PyTorch Version | Engine | Status |
|--------------|-----------------|---------|--------------|
| v0.22.0 | 2.7.1 | V1 only | ✅ Supported |
| v0.22.0 | 2.9.x | V1 only | ✅ Supported |

> **Note**: This plugin uses vLLM's V1 engine architecture. V0 engine is not supported.

Expand All @@ -57,7 +57,15 @@ The plugin leverages the following key components:
cd vllm-musa
```

2. Install vLLM Hardware Plugin for Moore Threads MUSA:
2. Install Python dependencies before installing `vllm-musa`. The MUSA
requirements entrypoint includes common dependencies and MUSA-private pins
such as `torch` and `torch_musa`.

```bash
pip install -r requirements/build.txt -r requirements/musa.txt
```

3. Install vLLM Hardware Plugin for Moore Threads MUSA:

```bash
# Standard installation (installs vLLM MUSA plugin and vLLM)
Expand All @@ -67,7 +75,7 @@ The plugin leverages the following key components:
pip install -e . --no-build-isolation -v
```

3. Verify the installation:
4. Verify the installation:

```bash
# Check plugin registration
Expand Down Expand Up @@ -105,6 +113,7 @@ Useful commands:

```bash
ccache --zero-stats
pip install -r requirements/build.txt -r requirements/musa.txt
pip install -e . --no-build-isolation -v
ccache --show-stats
```
Expand Down
13 changes: 10 additions & 3 deletions README_CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@

| vLLM 版本 | PyTorch 版本 | 引擎 | 状态 |
|-----------|--------------|---------|--------------|
| v0.22.0 | 2.7.1 | 仅 V1 | ✅ 已支持 |
| v0.22.0 | 2.9.x | 仅 V1 | ✅ 已支持 |

> **注意**:本插件使用 vLLM 的 V1 引擎架构,不支持 V0 引擎。

Expand All @@ -57,7 +57,14 @@
cd vllm-musa
```

2. 安装摩尔线程 MUSA 的 vLLM 硬件插件:
2. 安装 `vllm-musa` 前先安装 Python 依赖。MUSA requirements 入口包含通用依赖
和 `torch`、`torch_musa` 等 MUSA private 版本约束:

```bash
pip install -r requirements/build.txt -r requirements/musa.txt
```

3. 安装摩尔线程 MUSA 的 vLLM 硬件插件:

```bash
# 标准安装(安装 vLLM MUSA 插件和 vLLM)
Expand All @@ -67,7 +74,7 @@
pip install -e . --no-build-isolation -v
```

3. 验证安装:
4. 验证安装:

```bash
# 检查插件注册
Expand Down
47 changes: 47 additions & 0 deletions docker/build_image.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
#!/usr/bin/env bash
set -euo pipefail

SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
REPO_ROOT="$(cd "${SCRIPT_DIR}/.." && pwd)"
cd "${REPO_ROOT}"

PYTHON_VERSION="${PYTHON_VERSION:-3.10}"
BASE_IMAGE="${BASE_IMAGE:-ubuntu:22.04}"

MUSA_APT_SOURCE="${MUSA_APT_SOURCE:-https://dl.mthreads.com/repo/repository/ubuntu2204/}"
INSTALL_MUSA_STACK="${INSTALL_MUSA_STACK:-auto}"
MUSA_RUNTIME_VERSION="${MUSA_RUNTIME_VERSION:-5.1}"
MCCL_VERSION="${MCCL_VERSION:-2.3.0}"

PYTORCH_RELEASE="$(
awk -F'==' '/^torch==/ {gsub(/\.\*/, "", $2); print $2; exit}' \
requirements/musa_private.txt
)"
if [[ -z "${PYTORCH_RELEASE}" ]]; then
echo "Failed to derive PyTorch release from requirements/musa_private.txt" >&2
exit 1
fi

BUILD_MOONCAKE="${BUILD_MOONCAKE:-1}"
MOONCAKE_REPO="${MOONCAKE_REPO:-https://github.com/kvcache-ai/Mooncake.git}"
MOONCAKE_COMMIT="${MOONCAKE_COMMIT:-b6a841dc78c707ec655a563453277d969fb8f38d}"

IMAGE_REPOSITORY="${IMAGE_REPOSITORY:-vllm-musa}"
IMAGE_FLAVOR="${IMAGE_FLAVOR:-ubuntu22.04_py${PYTHON_VERSION}_musa_runtime_${MUSA_RUNTIME_VERSION}_pytorch_release_${PYTORCH_RELEASE}}"
IMAGE_TAG="${IMAGE_TAG:-${IMAGE_REPOSITORY}:${IMAGE_FLAVOR}}"

docker build \
--network host \
-f docker/musa.Dockerfile \
-t "${IMAGE_TAG}" \
--build-arg BASE_IMAGE="${BASE_IMAGE}" \
--build-arg PYTHON_VERSION="${PYTHON_VERSION}" \
--build-arg MUSA_APT_SOURCE="${MUSA_APT_SOURCE}" \
--build-arg INSTALL_MUSA_STACK="${INSTALL_MUSA_STACK}" \
--build-arg MUSA_RUNTIME_VERSION="${MUSA_RUNTIME_VERSION}" \
--build-arg MCCL_VERSION="${MCCL_VERSION}" \
--build-arg BUILD_MOONCAKE="${BUILD_MOONCAKE}" \
--build-arg MOONCAKE_REPO="${MOONCAKE_REPO}" \
--build-arg MOONCAKE_COMMIT="${MOONCAKE_COMMIT}" \
"$@" \
.
Loading