For TensorRT-LLM benchmarks ..whats the difference between batch_size and max_batch_size ? #1800
Unanswered
prasad-nair-amd
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
In trtllm-build tool there is an attribute --max_batch_size . What this attribute represent. Is this the same attribute as batch_size seen in other industry standard benchmarks. How to specify batch_size for a benchmark ?
Beta Was this translation helpful? Give feedback.
All reactions