Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ cargo add oar-ocr --features cuda,auto-download

This keeps the default `download-binaries` and `simd` features enabled, makes the ONNX Runtime CUDA execution provider available for selection, and downloads missing registered model files from ModelScope into `$OAR_HOME` when they are first used.

See the [Cargo feature guide](docs/features.md) for all available features and the [model guide](docs/models.md#auto-download-via-the-auto-download-feature) for model download and cache behavior.
See the [Cargo feature guide](docs/features.md) for all available features and the [model guide](docs/models.md#auto-download) for model download and cache behavior.

Builders also accept raw ONNX bytes such as `include_bytes!`, allowing models to be embedded in a single binary. See [Loading Models from Memory](docs/usage.md#loading-models-from-memory).

Expand Down Expand Up @@ -113,7 +113,7 @@ The classic pipeline runs ONNX models through ONNX Runtime and supports the foll

Available text-recognition checkpoints cover Chinese, Traditional Chinese, English, Arabic, Cyrillic, Devanagari, Greek, Eastern Slavic, Japanese, Georgian, Korean, Latin, Tamil, Telugu, and Thai scripts or languages.

### Vision-Language Models (VLM)
### Vision-Language Models

The [`oar-ocr-vl`](oar-ocr-vl/README.md) crate provides native [Candle](https://github.com/huggingface/candle) inference for compact document VLMs on CPU, CUDA, and Metal.

Expand Down
35 changes: 9 additions & 26 deletions docs/FAQ.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,29 +4,18 @@ Common build and runtime questions, distilled from GitHub issues.

## Windows: linking fails with `LNK2001: unresolved external symbol __std_find_trivial_8`

Unresolved `__std_*` symbols (`__std_find_trivial_*`, `__std_max_element_*`,
`__std_search_*`, ...) mean the linker's MSVC STL is too old. The prebuilt
ONNX Runtime binaries that `ort-sys` downloads are compiled with Visual
Studio 2022 (MSVC v143) and reference vectorized STL helpers that do not
exist in the VS 2019 (v142) link libraries.
Unresolved `__std_*` symbols (`__std_find_trivial_*`, `__std_max_element_*`, `__std_search_*`, ...) mean the linker's MSVC STL is too old. The prebuilt ONNX Runtime binaries that `ort-sys` downloads are compiled with Visual Studio 2022 (MSVC v143) and reference vectorized STL helpers that do not exist in the VS 2019 (v142) link libraries.

Fix:

1. Install **Visual Studio 2022** or **Build Tools for Visual Studio 2022**
with the "Desktop development with C++" workload (MSVC v143 + Windows SDK).
2. Run `cargo clean` and rebuild — rustc picks the newest installed MSVC
toolset automatically.
1. Install **Visual Studio 2022** or **Build Tools for Visual Studio 2022** with the "Desktop development with C++" workload (MSVC v143 + Windows SDK).
2. Run `cargo clean` and rebuild. Rustc picks the newest installed MSVC toolset automatically.

VS 2019 and VS 2022 build tools can coexist; only the newer one needs to be
present for linking. (See issue [#105](https://github.com/GreatV/oar-ocr/issues/105).)
VS 2019 and VS 2022 build tools can coexist. Only the newer one needs to be present for linking. See issue [#105](https://github.com/GreatV/oar-ocr/issues/105).

## GPU inference is slower than CPU for PP-OCRv6 tiny/small

Expected. The tiny/small models are so small that per-call overhead —
host↔device tensor copies, kernel launches, CPU/GPU synchronization, plus
pre/post-processing that always runs on the CPU — outweighs the compute the
GPU saves. Measured on an RTX 4090 + i9-13900KF (single image, warmup
excluded):
Expected. The tiny and small models are so compact that per-call overhead, including host-to-device tensor copies, kernel launches, CPU/GPU synchronization, and CPU pre/post-processing, can outweigh the computation saved by the GPU. The following results were measured on an RTX 4090 with an i9-13900KF using a single image and excluding warmup:

| Model | CPU | GPU (CUDA EP) |
| ------ | ---------- | ------------------------- |
Expand All @@ -37,13 +26,7 @@ excluded):
Guidelines:

- For tiny/small, use the default CPU mode (the `simd` feature is on by default).
- Use the medium model, or batch several images per `predict()` call, when
GPU acceleration matters.
- Exclude the first call when benchmarking: it includes cuDNN initialization
and algorithm selection (~5× slower than steady state).

Also note that requesting `OrtExecutionProvider::CUDA` without building with
`--features cuda` makes the pipeline builder return an error — check the
`Result` of `.build()`. Without the `cuda` feature, the downloaded ONNX
Runtime is CPU-only and the GPU is never used.
(See issue [#151](https://github.com/GreatV/oar-ocr/issues/151).)
- Use the medium model, or batch several images per `predict()` call, when GPU acceleration matters.
- Exclude the first call when benchmarking because it includes cuDNN initialization and algorithm selection (~5× slower than steady state).

Also note that requesting `OrtExecutionProvider::CUDA` without building with `--features cuda` makes the pipeline builder return an error. Check the `Result` of `.build()`. Without the `cuda` feature, the downloaded ONNX Runtime is CPU-only and the GPU is never used. See issue [#151](https://github.com/GreatV/oar-ocr/issues/151).
2 changes: 1 addition & 1 deletion docs/environment-variables.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ Runtime environment variables read by the oar-ocr crates. Project-specific varia

## `OAR_HOME`

Directory where the `auto-download` feature caches model files (default `~/.oar`). Bare model file names passed to the builders are downloaded here and verified against their expected SHA-256. See [models.md](models.md#auto-download-via-the-auto-download-feature) for the exact path resolution rules.
Directory where the `auto-download` feature caches model files (default `~/.oar`). Bare model file names passed to the builders are downloaded here and verified against their expected SHA-256. See [models.md](models.md#auto-download) for the exact path resolution rules.

```bash
OAR_HOME=/data/oar-models cargo run --release --example ocr -- doc.jpg
Expand Down
2 changes: 1 addition & 1 deletion docs/features.md
Original file line number Diff line number Diff line change
Expand Up @@ -159,7 +159,7 @@ If an execution provider feature is enabled, the supplied ONNX Runtime build mus
cargo add oar-ocr --features auto-download
```

Existing files and explicit paths remain under caller control. In-memory ONNX sources also bypass the download resolver. See the [model guide](models.md#auto-download-via-the-auto-download-feature) for the complete path-resolution and cache rules.
Existing files and explicit paths remain under caller control. In-memory ONNX sources also bypass the download resolver. See the [model guide](models.md#auto-download) for the complete path-resolution and cache rules.

`auto-download` supplies model files at runtime. It does not install ONNX Runtime or any accelerator dependencies.

Expand Down
Loading
Loading