You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When running the make evaluate command with the specified parameters, an error occurs indicating that the VLLMModelConfig initializer received an unexpected keyword argument.
Steps to Reproduce:
Set up the environment as per the project's guidelines.
Execute the following command:
make evaluate MODEL=deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B TASK=math_500
Expected Behavior: The evaluation process should complete without errors, utilizing the specified model and task parameters.
Actual Behavior: The following error message is displayed:
TypeError: VLLMModelConfig.__init__() got an unexpected keyword argument ''
Analysis: Upon investigation, it appears that the MODEL_ARGS variable in the Makefile's evaluate target may include an empty $(PARALLEL_ARGS) variable, leading to consecutive commas and resulting in an empty string being passed as a keyword argument.
Proposed Solution: Modify the MODEL_ARGS assignment in the Makefile to conditionally include $(PARALLEL_ARGS) only if it's not empty:
When running the
make evaluate
command with the specified parameters, an error occurs indicating that theVLLMModelConfig
initializer received an unexpected keyword argument.Steps to Reproduce:
Expected Behavior: The evaluation process should complete without errors, utilizing the specified model and task parameters.
Actual Behavior: The following error message is displayed:
Analysis: Upon investigation, it appears that the
MODEL_ARGS
variable in theMakefile
'sevaluate
target may include an empty$(PARALLEL_ARGS)
variable, leading to consecutive commas and resulting in an empty string being passed as a keyword argument.Proposed Solution: Modify the
MODEL_ARGS
assignment in theMakefile
to conditionally include$(PARALLEL_ARGS)
only if it's not empty:This change ensures that no extra comma is introduced when
$(PARALLEL_ARGS)
is empty, preventing the unintended empty string argument.The text was updated successfully, but these errors were encountered: