Skip to content

ROCm-Backend: added and tested docker image functionality with ROCm#1587

Open
siavashhub wants to merge 3 commits intolemonade-sdk:mainfrom
siavashhub:feat/docker-rocm-backend
Open

ROCm-Backend: added and tested docker image functionality with ROCm#1587
siavashhub wants to merge 3 commits intolemonade-sdk:mainfrom
siavashhub:feat/docker-rocm-backend

Conversation

@siavashhub
Copy link
Copy Markdown
Collaborator

Summary

  • Expands the install-selector tip for Vulkan to include a full ready-to-copy ROCm docker run
    command with the required --device=/dev/kfd, --device=/dev/dri, and --group-add video flags.
  • Adds an INFO log line in llamacpp_server.cpp that reports the selected backend at load time,
    making it easier to verify backend selection from logs.
  • Updates the LEMONADE_LLAMACPP_BACKEND environment variable to LEMONADE_LLAMACPP throughout
    documentation and Docker examples to match the recent changes.
  • Adds a dedicated "Docker Run with CPU backend" section to DOCKER_GUIDE.md.

Test plan

  • Verified Docker image starts correctly with LEMONADE_LLAMACPP=rocm on an AMD GPU host
  • Confirmed GPU utilization via rocm-smi during inference
  • Confirmed LEMONADE_LLAMACPP=cpu and LEMONADE_LLAMACPP=vulkan still work as expected
  • Checked log output shows "Using LlamaCpp Backend: rocm/vulkan" on model load
  • Reviewed install-selector rendering for the Vulkan GPU path in a browser

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant