Hi! First of all, thanks for the package. I'm building a package on top of it designed to give the user control over the context: for instance, the user could at any point in the chat, select several turns in the conversation so far and replace them with a summary of their contents (to cut the context window down). Or, perhaps, just drop them. Or mask/summarize prior tool outputs. And so forth.
I've read your caution about llm-chat-prompt-interactions and it pretty-much blows my intended approach out of the water. Have you thought about this use case, or would it be fair to say that the llm package just doesn't offer this point of extensibility? Perhaps you could define a new model capability (something alongside `'streaming') that would indicate whether a given model can handle having its prior interactions manipulated like this.
Hi! First of all, thanks for the package. I'm building a package on top of it designed to give the user control over the context: for instance, the user could at any point in the chat, select several turns in the conversation so far and replace them with a summary of their contents (to cut the context window down). Or, perhaps, just drop them. Or mask/summarize prior tool outputs. And so forth.
I've read your caution about llm-chat-prompt-interactions and it pretty-much blows my intended approach out of the water. Have you thought about this use case, or would it be fair to say that the
llmpackage just doesn't offer this point of extensibility? Perhaps you could define a new model capability (something alongside `'streaming') that would indicate whether a given model can handle having its prior interactions manipulated like this.