High token loop hallucination rate, high no response rate

[iDynbek]

Problems:
Model often hallucinates with token loop sections.
Model often returns empty responses.

Reproduction:
Russian/Kazakh language scanned texts.

Possible solutions:

1. Provide more detailed guidance on how to use the model with settings, preprocessing steps.
2. Fine-tune the model on synthetic dataset, as I am not aware of existence of any annotated Kazakh/Russian datasets for OCR readily available.

I've tried to integrate chandra repo code into my pipeline with exact vllm settings, request settings, prompts and preprocessing, but still face with high error and hallucination rate that makes this model barely usable in production in my language set. In my test set I have ~60 documents and 5-6 of them on average always have token loop hallucination. 1 or 2 documents consistently have missing pages.

Scans I use are of decent quality with clearly recognizable text, but the model still fails at relatively high rate.

I also tried sending pdf images directly without intermediate conversion steps, hoping that lossless pipeline will help the situation, but no success.

Can't share the exact documents bc they are private.
Any help could be appreciated.

[mendeel]

Try to play with the temperature and repeat penalty parameters. In my tests using LM Studio, I'm using 0.25 temperature and I upped repeat penalty from 1,1 to 1,25 giving me better results regarding token loops.

[iDynbek]

Thanks for the hint. But now I can't say much since I have moved to Qwen3-VL-32B. It is more precise on my tests, and it also suffers from token loops.

I played with repeat penalty on it, but for some reason it starts to substitute Cyrillic characters with Latin ones. I suspect that Chandra can also have similar issues (but I can't test it right now).