Skip to content

✨ Feat: Add Dockerfile.vllm to support model serving with vLLM#12

Open
bglar wants to merge 4 commits intomainfrom
feat-extend-base-img-with-vllm
Open

✨ Feat: Add Dockerfile.vllm to support model serving with vLLM#12
bglar wants to merge 4 commits intomainfrom
feat-extend-base-img-with-vllm

Conversation

@bglar
Copy link
Contributor

@bglar bglar commented Jan 29, 2026 •
edited
Loading

Summary

Add Dockerfile.vllm to support model serving with vLLM

Related Issue (optional)

How to test this PR?

  1. Port-forward to deployed pod:
    kubectl port-forward -n geospatial-dev deployment/vllm-geospatial-studio 8000:8000

  2. Send POST request to /pooling endpoint:

   {
       "data": {
           "data": "https://ibm.box.com/shared/static/orraqbi379pwfpvjaurufddfj2qjftly.tif",
           "data_format": "url",
           "out_data_format": "path",
           "out_path": "/data/f32cb5c9-6c6b-426d-8feb-a901bf55a411/f32cb5c9-6c6b-426d-8feb-a901bf55a411-task_0",
           "image_format": "tiff"
       },
       "model": "ibm-nasa-geospatial/Prithvi-EO-2.0-300M-TL-Sen1Floods11"
   }

Screenshots / Logs (optional)

Checklist

  • This PR targets the main branch
  • I have added or updated relevant docs.
  • I have not included any secrets or credentials.
  • Linting and formatting checks pass.

@bglar bglar force-pushed the feat-extend-base-img-with-vllm branch from a6b0459 to 6f939ee Compare January 29, 2026 13:45
@bglar bglar marked this pull request as ready for review January 30, 2026 07:07
@bglar bglar force-pushed the feat-extend-base-img-with-vllm branch from 1bdde8f to 1ce392c Compare January 30, 2026 07:08
Beldine-Moturi pushed a commit that referenced this pull request Feb 2, 2026
@cwachira
Merge pull request #12 from terrastackai/cwachira-patch-1
324300f 
Dockerfiles optimization
@bglar bglar force-pushed the feat-extend-base-img-with-vllm branch from 1ce392c to 30aec7f Compare March 24, 2026 11:23
@bglar bglar force-pushed the feat-extend-base-img-with-vllm branch from 30aec7f to acfea59 Compare March 24, 2026 11:28
@bglar
Create a dedicated dockerfile for vllm inference
33326d4 
Signed-off-by: Brian Glar <bogolla4@gmail.com>
@bglar bglar force-pushed the feat-extend-base-img-with-vllm branch from 01153af to 33326d4 Compare March 24, 2026 11:33
@bglar
fix(vllm): resolve permission and module errors in Docker image
9ef2660 
- Use full python3 package instead of python3-minimal for ctypes support
- Preserve numpy test directories during cleanup
- Create cache/config directories with proper permissions

Signed-off-by: Brian Glar <bogolla4@gmail.com>
@bglar bglar changed the title ✨ Feat: Extend terratorch image with vllm support ✨ Feat: Add Dockerfile.vllm to support model serving with vLLM Mar 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@bedwards-ibm bedwards-ibm Awaiting requested review from bedwards-ibm bedwards-ibm is a code owner

@cwachira cwachira Awaiting requested review from cwachira cwachira is a code owner

@fredotieno fredotieno Awaiting requested review from fredotieno fredotieno is a code owner

@WanjiruCate WanjiruCate Awaiting requested review from WanjiruCate WanjiruCate is a code owner

At least 2 approving reviews are required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@bglar