multi-lora batching

[Diffizle]

Does vllm support batching of prompts with different lora-adapters? Is there a more detailed example code?
the example shown in <examples/offline_inference/multilora_inference.py> does not seem to express this feature

[jeejeelee]

Does vllm support batching of prompts with different lora-adapters?

vLLM support this featrue

the example shown in <examples/offline_inference/multilora_inference.py> does not seem to express this feature

multilora_inference.py show this feature, different lora_id to represent different LoRAs.

[Diffizle]

yeah, I also figured it out with debugging
when max_loras is set to > 1, it is indeed batching loras
thanks for comment !