Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
156 changes: 156 additions & 0 deletions tests/functional/test_vllm/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,156 @@
# vLLM Dockerfile Tests

Validates that the vLLM Docker environment builds correctly and contains all required components.

## Overview

This test suite ensures the vLLM Dockerfile:
- ✅ Builds successfully without errors
- ✅ Contains required components (Python 3.12, vLLM dependencies)
- ✅ Runs with proper security (non-root user)

## Test Structure

```
test_vllm/
├── test_dockerfile.py # 3 tests
└── README.md # This file
```

### Tests

**TestVLLMDockerBuild** - 3 comprehensive tests (requires Docker):

1. `test_image_builds_successfully` - Validates Docker build completes
2. `test_image_has_required_components` - Verifies Python 3.12, virtualenv, paths
3. `test_container_runs_as_non_root` - Security validation (uid 1000)

## Running Tests

### As Part of All Functional Tests (Recommended):
```bash
make functional-tests # Runs all functional tests including vLLM
```

### Directly with pytest:
```bash
# Run only vLLM tests
pytest tests/functional/test_vllm/test_dockerfile.py -v

# Or as part of all functional tests
pytest tests/functional/ -v
```

## CI/CD Integration

### Jenkins
```groovy
stage('Test vLLM') {
steps {
sh 'pytest tests/functional/test_vllm/test_dockerfile.py -v'
}
}
```

### GitHub Actions
```yaml
- name: Test vLLM Dockerfile
run: pytest tests/functional/test_vllm/test_dockerfile.py -v
```

### GitLab CI
```yaml
test:vllm:
script:
- pytest tests/functional/test_vllm/test_dockerfile.py -v
```

## Performance

| Run Type | Time |
|----------|------|
| First run (with image download) | 5-15 min |
| Cached run | 3-5 min |

The Docker image is built once and reused across all tests in the class.

## Requirements

### On Your Host Machine (to run tests):
- Docker daemon running
- Python 3.8+ (for pytest test runner)
- pytest, docker Python packages

### Inside the Docker Container (what gets tested):
- Python 3.12 (installed by Dockerfile)
- vLLM dependencies (from base image)
- ~15GB free disk space for image

## Troubleshooting

### Docker not running:
```bash
# Error: "Docker is not available or accessible"
# Solution:
# macOS/Windows: Start Docker Desktop
# Linux: sudo systemctl start docker
```

### Insufficient disk space:
```bash
# Error: "no space left on device"
# Solution:
docker system prune -a # Remove unused images
docker images # Check current images
df -h # Check disk space
```

### Build fails - Base image unavailable:
```bash
# Error: "pull access denied" or "manifest unknown"
# Context: Image pulled from Docker Hub: vllm/vllm-openai:v0.11.0
# Solution:
# 1. Check internet connection
# 2. Verify base image exists: docker pull vllm/vllm-openai:v0.11.0
# 3. If behind corporate proxy, configure Docker proxy settings
# 4. If authentication required, login: docker login
```

### Build fails - Authentication required:
```bash
# Error: "pull access denied for vllm/vllm-openai" with "authentication required"
# Solution:
# 1. Login to Docker Hub: docker login
# 2. Or use credentials: docker login -u <username> -p <password>
# 3. If using private registry, configure credentials in Docker config
```

### Build fails - Dependency installation:
```bash
# Error: "E: Unable to locate package python3.12"
# Context: Dockerfile specifies Python 3.12 installation (line 4)
# Solution:
# 1. Verify base image hasn't changed: docker pull vllm/vllm-openai:v0.11.0
# 2. Check if Dockerfile was modified in public_dropin_gpu_environments/vllm/
# 3. Try rebuilding without cache: docker build --no-cache
```

### Container fails to start:
```bash
# Error: "Container did not start in time"
# Solution:
# 1. Check Docker daemon logs: docker logs <container-id>
# 2. Verify sufficient resources (CPU/memory)
# 3. Check for port conflicts
```


## Design Philosophy

These tests follow best practices:
- **No redundancy**: Each test validates something unique
- **Fast feedback**: Docker build happens once, reused across tests
- **CI-friendly**: Integrates seamlessly with existing test infrastructure
- **Clear failures**: Detailed error messages for debugging


111 changes: 111 additions & 0 deletions tests/functional/test_vllm/test_dockerfile.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
"""
Copyright 2025 DataRobot, Inc. and its affiliates.
All rights reserved.
This is proprietary source code of DataRobot, Inc. and its affiliates.
Released under the terms of DataRobot Tool and Utility Agreement.

vLLM Dockerfile build and validation tests.

These tests validate that the vLLM Docker image can be built successfully
and contains all required components for running in production.
"""

import os
import time

import docker
import pytest

from tests.constants import PUBLIC_DROPIN_GPU_ENVS_PATH


class TestVLLMDockerBuild:
"""Docker build tests for vLLM environment (requires Docker)."""

VLLM_PATH = os.path.join(PUBLIC_DROPIN_GPU_ENVS_PATH, "vllm")
IMAGE_TAG = "vllm-test:latest"

@pytest.fixture(scope="class")
def docker_client(self):
"""Provides a Docker client, failing if Docker is not available."""
try:
client = docker.from_env()
client.ping()
return client
except Exception as e:
pytest.fail(f"Docker is not available or accessible: {e}")

@pytest.fixture(scope="class")
def vllm_image(self, docker_client):
"""Builds the vLLM Docker image once per test class."""
try:
image, build_logs = docker_client.images.build(
path=self.VLLM_PATH,
tag=self.IMAGE_TAG,
rm=True,
forcerm=True,
)
except docker.errors.BuildError as e:
build_log = "".join([log.get("stream", "") for log in e.build_log or []])
pytest.fail(
f"Docker build failed: {e}\nBuild Log (last 50 lines):\n{build_log[-5000:]}"
)

yield image

# Cleanup: remove the image after tests
try:
docker_client.images.remove(image.id, force=True)
except docker.errors.APIError:
pass # Ignore errors during cleanup

@pytest.fixture(scope="class")
def vllm_container(self, docker_client, vllm_image):
"""Creates and runs a container that is shared across tests in this class."""
container = docker_client.containers.run(
vllm_image.id,
command="sleep 600", # Keep container running for tests
detach=True,
)
# Wait for the container to be in the 'running' state
for _ in range(10):
container.reload()
if container.status == "running":
break
time.sleep(1)
else:
pytest.fail("Container did not start in time")

yield container
# Reliable cleanup
try:
container.remove(force=True)
except docker.errors.APIError:
pass # Ignore errors during cleanup

def test_image_builds_successfully(self, vllm_image):
"""Verify the Docker image was built and has an ID."""
assert vllm_image is not None
assert vllm_image.id is not None

def test_image_has_required_components(self, vllm_container):
"""Verify the built image contains required paths and packages."""
required_paths = ["/opt/code", "/opt/venv", "/opt/.home"]
for path in required_paths:
exit_code, _ = vllm_container.exec_run(f"test -e {path}")
assert exit_code == 0, f"Required path not found in container: {path}"

# Check for Python 3.12
exit_code, _ = vllm_container.exec_run("python3.12 --version")
assert exit_code == 0, "Python 3.12 not found in container"

# Check that virtualenv was created
exit_code, _ = vllm_container.exec_run("/opt/venv/bin/python --version")
assert exit_code == 0, "Virtualenv Python not found in container"

def test_container_runs_as_non_root(self, vllm_container):
"""Verify the container runs with the correct non-root user."""
exit_code, output = vllm_container.exec_run("whoami")
assert exit_code == 0
# The user is 'datarobot' in the Dockerfile
assert output.decode().strip() == "datarobot"