fix: robust LLM JSON parsing + sanitize report sections + chat history dedup (closes #624 #622 #601 #599 #577)

[rqd6f4g6zn-bit]

Fix 4 open bugs: robust LLM JSON parsing + sanitize report sections + chat history dedup

Closes #624, #622, #601, #599, #577

Summary

This PR addresses four open bugs that all stem from how the backend handles slightly-non-conforming LLM outputs (trailing text, raw tool_call JSON, duplicate user messages). Net change: backend is more defensive, no behavioral regressions on conforming outputs.

Issues fixed

#624 / #622 / #601 — Unexpected non-whitespace character after JSON at position N + 500 on /api/graph/ontology/generate

Root cause: chat_json() in backend/app/utils/llm_client.py runs strict json.loads() after stripping Markdown fences. Many LLMs (qwen-plus, gemma, ollama-served models) append trailing prose after the JSON block even with response_format=json_object — strict parsing fails with the reported error.

Fix: Extracted parsing into _parse_llm_json() with a 5-stage strategy:

1. Strip Markdown fences (existing)
2. Strict json.loads (fast path)
3. json.JSONDecoder().raw_decode() — parses JSON prefix, ignores trailing text (logs a warning)
4. Balanced-brace extraction (handles leading prose + embedded JSON)
5. Strip control chars + retry

Falls through to a ValueError with a clear preview if everything fails. Logs visibility into which strategy succeeded so we can monitor which LLMs misbehave.

Tests: 8 unit-test cases covering clean JSON, fenced JSON, trailing text, fences+trailing, leading prose, nested-objects+trailing, control chars, and the exact failure mode at position 10243 — all pass.

#599 — Section content is raw unexecuted tool_call JSON

Root cause: In backend/app/services/report_agent.py::write_section(), three code paths can return a final_answer that consists entirely of an unexecuted tool_call JSON (e.g. {"name":"quick_search","parameters":{...}}):

• Final Answer path (line ~1392) — when LLM prefixes raw JSON with Final Answer:
• "No prefix" fallback (line ~1491) — when LLM has used enough tools but emits a tool_call instead of prose
• Force-finalize path (line ~1519) — when max iterations hit and LLM still emits a tool_call

All three paths now go through new _sanitize_section_content() which detects the leak pattern (full content parses as JSON and name matches VALID_TOOL_NAMES) and replaces with a clear fallback message instead of leaking the raw artifact.

#577 — Report-Agent chat repeats first answer regardless of follow-up question

Likely root cause: Frontend can (under some flows) include the just-sent user message inside the chat_history array. Backend then appends the same message again at the end → LLM sees duplicate-trailing user message and ignores the latest one, returning the prior answer style.

Fix: In report_agent.py::chat(), defensively:

• Only keep {role, content} from history items (defensive against extra fields)
• Skip entries that match the current message
• Added debug log of constructed messages array for future diagnostics

This is defensive — even if the frontend filters correctly, this prevents regressions from future call sites.

Files changed

backend/app/utils/llm_client.py       +112 -10  (new _parse_llm_json with 5 strategies)
backend/app/services/report_agent.py   +50  -10  (new _sanitize_section_content + 3 call sites; chat() history dedup)

Verification

• All unit tests for _parse_llm_json pass (8/8 cases covering each strategy)
• Backend starts cleanly: uv run python run.py → all routes return correct validation errors on empty bodies (400/404, no more 500s on validation failures)
• No new dependencies
• No changes to public API contracts

Notes for maintainers

• The JSON parser now emits warnings via logger when fallback strategies are used. Monitor these — high frequency from a specific model suggests adjusting that model's system prompt.
• _sanitize_section_content() returns a German-language fallback message ("_(Hinweis: Für diesen Abschnitt..."). If the project wants Chinese/English, localize via the i18n system in app/utils/locale.py.
• For Bug for the last step among the five steps! #577 a fuller fix would be on the frontend (filter logic in Step5Interaction.vue::sendToReportAgent) — but the backend defensive fix is independent and prevents regressions.

[Thamer26]

So what do I do to actually make it work?

[hse00435-hub]

Thanks for the quick patch in #626! However, please note that the live production website is STILL throwing the exact same error (as shown in my newly uploaded screenshot).

It seems the fix has been coded and passed your local unit tests, but it hasn't been merged or deployed to the live production server yet. Paid users are still completely blocked by this position 10243 bug on the web platform.

Could you please nudge the team to deploy PR #626 to the production server ASAP? Thank you!

[mirofx313-eng]

Thanks for the quick patch in #626! However, please note that the live production website is STILL throwing the exact same error (as shown in my newly uploaded screenshot).

It seems the fix has been coded and passed your local unit tests, but it hasn't been merged or deployed to the live production server yet. Paid users are still completely blocked by this position 10243 bug on the web platform.

Could you please nudge the team to deploy PR #626 to the production server ASAP? Thank you!

Its 01/06/2026 and the issue is still present is anyone going to do anything with the issue? Paid subscribesr are locked out of the tool.