Conversion Activation Error (Aggregation Error org.springframework.web.reactive.function.client.WebClientResponseException: 413 Payload Too Large from POST https://api.groq.com/openai/v1/chat/completions)

[aniwange33]

I’m currently working on activating conversation features in my Spring AI application but encountered a significant error. When I attempt to enable conversation functionality using ChatMemory, I receive the following error:

"*2024-11-05T10:57:17.441+01:00 ERROR 44072 --- [Bip.it] [oundedElastic-4] o.s.ai.chat.model.MessageAggregator : Aggregation Error

org.springframework.web.reactive.function.client.WebClientResponseException: 413 Payload Too Large from POST https://api.groq.com/openai/v1/chat/completions*"

This issue arises when I include the following code in RagService:

public RagService(ChatClient.Builder chatBuilder, VectorStore vectorStore) {

ChatMemory chatMemory = new InMemoryChatMemory();

this.chatClient = chatBuilder

.defaultAdvisors(

new MessageChatMemoryAdvisor(chatMemory),

new QuestionAnswerAdvisor(vectorStore),

new SimpleLoggerAdvisor())

.build();

}

[asaikali]

How many messages are in the history before are getting this error message. I have created issue #1704 as a generalized version of what you are asking for.

@aniwange33 can you please us know if you got around your issues by reducing the amount of message in the chat memory, can you provide a sample reproducer of the issue.

[aniwange33]

How many messages are in the history before are getting this error message. I have created issue #1704 as a generalized version of what you are asking for.

@aniwange33 can you please us know if you got around your issues by reducing the amount of message in the chat memory, can you provide a sample reproducer of the issue.

I couldn’t log the number of messages in the history without getting an error. However, when I used Filter.Expression, the error didn’t occur. Despite this, the conversation activation hasn’t been working smoothly, and sometimes the responses are inaccurate.

public RagService(ChatClient.Builder chatBuilder, VectorStore vectorStore, @value("classpath:/system.md") Resource systemPrompt) {
ChatMemory chatMemory = new InMemoryChatMemory();
var fb = new FilterExpressionBuilder();
Filter.Expression source = fb.eq("source", "FY25-MER-2.8-Indicator-Reference-Guide.pdf").build();
SearchRequest searchRequest = SearchRequest.defaults().withTopK(2).withFilterExpression(source);
List advisors = List.of(
new PromptChatMemoryAdvisor(chatMemory),
new QuestionAnswerAdvisor(vectorStore, searchRequest),
new SimpleLoggerAdvisor()
);
this.chatClient = chatBuilder.defaultSystem(systemPrompt).defaultAdvisors(advisors).build();
}

public Flux streamChatWithConversationEnable(QueryRequest request) {
log.info("message {}", request.getQuery());
Parser parser = Parser.builder().build();
HtmlRenderer renderer = HtmlRenderer.builder().build();
return Flux.defer(() ->
chatClient.prompt()
.user(request.getQuery())
.advisors(spec -> spec
.param(CHAT_MEMORY_CONVERSATION_ID_KEY, request.getConversationId())
.param(CHAT_MEMORY_RETRIEVE_SIZE_KEY, 2)
)
.stream().content()
.bufferUntil(s -> s.endsWith(" "))
.map(list -> {
StringBuilder contentBuilder = new StringBuilder();
list.forEach(contentBuilder::append);
return contentBuilder.toString();
})
.map(content -> {
String adjustedContent = content
.replaceAll(" - ", "\n - ")
.replaceAll("(?i)(\bNote:\b|\bWarning:\b|\bImportant:\b)", "$1")
.replaceAll("(?i)\b(Note|Warning):\b", "$1:")
.replaceAll("\s*\n{2,}(?![^()]*\))", "\n\n- ");
Node document = parser.parse(adjustedContent);
String render = renderer.render(document);
BipChatResponse chatResponse = new BipChatResponse();
chatResponse.setConversationId(request.getConversationId());
chatResponse.setResponseContent(render); // Set plain text instead of HTML
return chatResponse;
})
);
}