Skip to content

Commit b334102

Browse files
authored
[https://nvbugs/5564465][fix] Overwrite only if default_max_tokens is legal (#8538)
Signed-off-by: Pengyun Lin <[email protected]>
1 parent db3c373 commit b334102

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

tensorrt_llm/executor/base_worker.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -432,7 +432,7 @@ def _deduce_max_tokens(request: GenerationRequest,
432432
# default_max_tokens is the biggest available value
433433
if max_tokens is None:
434434
return default_max_tokens
435-
elif max_tokens > default_max_tokens:
435+
elif max_tokens > default_max_tokens and default_max_tokens > 0:
436436
logger.warning(
437437
f"User-specified `max_tokens` ({max_tokens}) is greater than deduced "
438438
f"`default_max_tokens` ({default_max_tokens}), using default_max_tokens instead."

0 commit comments

Comments
 (0)