The query is exceeding the maximum allowed number of tokens #288
Unanswered
erikboberg
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Thanks for an awsome software!
When asking my collections I sometimes get messages like this:
Total tokens = 14513, prompt tokens = 13513, completion tokens = 1000
The query is exceeding the maximum allowed number of tokens
That makes no sense since I am hosting my LLM locally and it has a context window of 128000.
I am using a openai compatible endpoint, hosted LLM via llama.cpp.
Why am I seeing this? Is it affecting the quality of responses?
Beta Was this translation helpful? Give feedback.
All reactions