Skip to content

Commit

Permalink
Show debug log, not warning when use default tokenizer for context st…
Browse files Browse the repository at this point in the history
…uffing
  • Loading branch information
debanjum committed Sep 13, 2024
1 parent f173188 commit 178b78f
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion src/khoj/processor/conversation/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -238,7 +238,7 @@ def truncate_messages(
else:
encoder = AutoTokenizer.from_pretrained(default_tokenizer)
state.pretrained_tokenizers[default_tokenizer] = encoder
logger.warning(
logger.debug(
f"Fallback to default chat model tokenizer: {tokenizer_name}.\nConfigure tokenizer for unsupported model: {model_name} in Khoj settings to improve context stuffing."
)

Expand Down

0 comments on commit 178b78f

Please sign in to comment.