Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error in make evaluate: TypeError: VLLMModelConfig.__init__() got an unexpected keyword argument '' #163

Open
mattdepaolis opened this issue Feb 2, 2025 · 0 comments · May be fixed by #246

Comments

@mattdepaolis
Copy link

When running the make evaluate command with the specified parameters, an error occurs indicating that the VLLMModelConfig initializer received an unexpected keyword argument.

Steps to Reproduce:

  1. Set up the environment as per the project's guidelines.
  2. Execute the following command:
make evaluate MODEL=deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B TASK=math_500

Expected Behavior: The evaluation process should complete without errors, utilizing the specified model and task parameters.

Actual Behavior: The following error message is displayed:

TypeError: VLLMModelConfig.__init__() got an unexpected keyword argument ''

Analysis: Upon investigation, it appears that the MODEL_ARGS variable in the Makefile's evaluate target may include an empty $(PARALLEL_ARGS) variable, leading to consecutive commas and resulting in an empty string being passed as a keyword argument.

Proposed Solution: Modify the MODEL_ARGS assignment in the Makefile to conditionally include $(PARALLEL_ARGS) only if it's not empty:

MODEL_ARGS="pretrained=$(MODEL),dtype=float16$(if $(PARALLEL_ARGS),,$(PARALLEL_ARGS)),max_model_length=32768,gpu_memory_utilisation=0.8"

This change ensures that no extra comma is introduced when $(PARALLEL_ARGS) is empty, preventing the unintended empty string argument.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant