-
Notifications
You must be signed in to change notification settings - Fork 14.7k
common : add specific warning for multiple -m options #19113
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
This commit updates the deprecation warning for multiple model options with a specific message that does not mention comma-separated values as an option. The motivation for this is that currently it can be confusing if multiple model options are specified with llama-server as this would display the following warning: ```console DEPRECATED: argument '--model' specified multiple times, use comma-separated values instead (only last value will be used) ``` But llama-server does not support a comma-separated models option. With this change the user is informed that multiple models can be supported in router mode if using llama-server: ```console DEPRECATED: multiple --model options are not supported (only last value will be used). Use router mode for llama-server (see --help) ```
|
Unless I missed something, we never support multiple -m, so using the word "deprecated" here make one thinks that it used to be, but now no longer supported |
Yeah good point, and I was not sure about using "deprecated" here but thought I'd perhaps it was supported at some point that I was not aware of. Let me know and I'll change this to something else, perhaps just |
|
But to start, in which case people may specify multiple I personally believe that multiple |
This was suggested to me by an AI actually (can't recall which model) which is why I looked into this as I did not think this was supported.
I agree, and if that is alright to do that here it would be preferred. |
In such case, I think there is no need to change anything at all. If something need to changed, I expect to throw an error on any args that are not supposed to have multiple values. We have plenty of them: |
This commit updates the argument parsing to check for valid multiple value options, for example using --file --file with llama-completions is something that is supported and this is the case for a few other options as well. But for the majority of options specifying them multiple times is not supported and this will now thow an error. For example using multiple --model options will now result in an error: ```console $ llama-completion -m models/Qwen2.5-0.5B-Instruct.gguf --model models/Qwen2.5-0.5B-Instruct.gguf error: argument '--model' cannot be specified multiple times ``` But using multiple --file options will only result in a warning: ```console $ llama-completion -m models/Qwen2.5-0.5B-Instruct.gguf -f test_file.txt -f test_file.txt DEPRECATED: argument '-f' specified multiple times, use comma-separated values instead (only last value will be used) ... ```
I actually think this is a reason to change this as it might be suggested to users, and it is somewhat confusing as it is right now. But I agree that throwing an error for all the options that don't actually support being specified multiple times would be nicer. I'll add a commit with a suggestion for this: For example, using multiple --model options would result in an error: $ llama-completion -m models/Qwen2.5-0.5B-Instruct.gguf --model models/Qwen2.5-0.5B-Instruct.gguf
error: argument '--model' cannot be specified multiple timesBut using multiple --file options would only result in a warning (like it currently does): $ llama-completion -m models/Qwen2.5-0.5B-Instruct.gguf -f test_file.txt -f test_file.txt
DEPRECATED: argument '-f' specified multiple times, use comma-separated values instead (only last value will be used)
... |
This commit updates the deprecation warning for multiple model options with a specific message that does not mention comma-separated values as an option.
The motivation for this is that currently it can be confusing if multiple model options are specified with llama-server as this would display the following warning:
DEPRECATED: argument '--model' specified multiple times, use comma-separated values instead (only last value will be used)But llama-server does not support a comma-separated models option.
With this change the user is informed that multiple models can be supported in router mode if using llama-server:
DEPRECATED: multiple --model options are not supported (only last value will be used). Use router mode for llama-server (see --help)