Question: How to handle too large pages?

[Sebobo]

We get the following a status error alert on some pages, with the quest response returning:

{
    "status": false,
    "error": "This model's maximum context length is 16385 tokens. However, your messages resulted in 68396 tokens. Please reduce the length of the messages."
}

First, the error should also be shown as flash message, and not just an alert without a message, see #2.

Second: How can this be resolved from an integrators perspective?

We probably could strip a few more parts of a page from the rendered markup before markdown conversion.

[Sebobo]

OK some problem is that we render lots of metadata in the content for backend usage and to help editors, which isn't stripped.
I'll how far we get, wenn I strip that.

[Sebobo]

So by removing the metadata I got down to 16700 characters. So I modified the prompt to crop the content {String.cropAtSentence(content, 15500)}.

[sbruggmann]

@Sebobo #2 is probably the most important task here.
Depending on the model there's a big difference in context size and by optimizing the prompt context you worked on worked on the correct spot.

I'd love to have a compressor which would remove not needed content automatically, based on the prompt (like cursor or code are doing it).

[Sebobo]

I also think documentation is enough. So f.e. adding an example on how to crop in the correct place, as I also first had to think a bit where the correct place is to crop.