Will flash attn reduce the quality of OCR text ?

[Greatz08]

I didnt saw the native support for flash attn in this project and so i was wondering if flash attn is worth it or not. I also had this question in my mind like does it reduce text recognition or create more issues for text recognition because of which you didnt add support for it

[GreatV]

Hi @Greatz08, Flash Attention should not reduce OCR text recognition quality.

In oar-ocr, Flash Attention is not used by all OCR paths. Whether it is enabled or not only affects the vl module, and does not affect the other OCR modules. The main reason we have not enabled Flash Attention by default is installation and build stability. Flash Attention usually requires extra native/CUDA compilation and is sensitive to the user's GPU, CUDA toolkit, compiler, and target platform. This can easily introduce build failures or compatibility issues, which is not ideal for the default experience of this crate.

For the vl module, Flash Attention is mainly a performance/memory optimization rather than a requirement for correctness. We may consider it as an optional acceleration feature later, but we do not want it to be a required dependency.