Skip to content

Commit 04aafa9

Browse files
authored
docs: Update trtllm multinode multimodal docs (#3369)
Signed-off-by: Indrajit Bhosale <[email protected]>
1 parent ff7d43d commit 04aafa9

File tree

1 file changed

+21
-0
lines changed

1 file changed

+21
-0
lines changed

components/backends/trtllm/multinode/multinode-multimodal-example.md

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,27 @@ limitations under the License.
1717

1818
# Example: Multi-node TRTLLM Workers with Dynamo on Slurm for multimodal models
1919

20+
> [!IMPORTANT]
21+
> There are some known issues in tensorrt_llm==1.1.0rc5 version for multinode multimodal support. It is important to rebuild the dynamo container with a specific version of tensorrt_llm commit to use multimodal feature.
22+
>
23+
> **Build Container**
24+
> ```bash
25+
> ./container/build.sh --framework trtllm --tensorrtllm-commit b4065d8ca64a64eee9fdc64b39cb66d73d4be47c
26+
> ```
27+
>
28+
> **Run Container**
29+
> ```bash
30+
> ./container/run.sh --framework trtllm -it
31+
> ```
32+
>
33+
> **Update Engine Configuration Files**
34+
>
35+
> Before running the deployment, you must update the engine configuration files to change `backend: DEFAULT` to `backend: default` (lowercase). Run the following command:
36+
> ```bash
37+
> sed -i 's/backend: DEFAULT/backend: default/g' /mnt/engine_configs/multimodal/llama4/prefill.yaml /mnt/engine_configs/multimodal/llama4/decode.yaml
38+
> ```
39+
40+
2041
This guide demonstrates how to deploy large multimodal models that require a multi-node setup. It builds on the general multi-node deployment process described in the main [multinode-examples.md](./multinode-examples.md) guide.
2142
2243
Before you begin, ensure you have completed the initial environment configuration by following the **Setup** section in that guide.

0 commit comments

Comments
 (0)