Add HunyuanOCR 1.5 DFlash decoding

[GreatV]

What changed

• add a native Candle implementation of the HunyuanOCR 1.5 DFlash draft model
• collect target features after decoder boundaries 1, 8, 15, and 22
• cache draft context K/V incrementally and draft 15 mask tokens in parallel
• verify candidate blocks with the target, accept the matching prefix, and roll target KV back after rejection
• add HunyuanOcr::from_dirs, from_dir_with_dflash, and --dflash-dir
• add benchmark-mode token counts and deterministic fingerprints to the HunyuanOCR example
• report HunyuanOCR 1.5 as 1.0B in the model table

Why

The official HunyuanOCR 1.5 checkpoint includes a DFlash draft under dflash/, but the Rust backend previously only supported autoregressive target decoding.

During integration, block verification also exposed a long-context causal-mask bug: mask position comparisons were performed in BF16, which cannot distinguish adjacent positions at document-scale offsets. The comparison now stays in integer space before producing the model-dtype additive mask.

Impact

Single-document HunyuanOCR 1.5 calls can opt into the official 15-token DFlash path. Existing calls and multi-image batches retain the autoregressive path.

On the existing 30-page English OmniDocBench v1.5 sample list, RTX 4090, CUDA, identical prompt and max_tokens=256:

• AR: 178.83 s, 33.50 tokens/s, 5,991 emitted tokens
• DFlash: 144.75 s, 42.12 tokens/s, 6,097 emitted tokens
• end-to-end latency: 19.1% lower
• emitted-token throughput: 25.7% higher
• 17/30 page token fingerprints were bit-identical; BF16 batched verification can take a different greedy path from one-token AR because operation ordering changes rounding

The short text-recognition smoke case was token-identical after the integer causal-mask fix.

Validation

• cargo test -p oar-ocr-vl --features download-binaries --lib (95 passed)
• cargo check -p oar-ocr-vl --all-targets --features cuda,download-binaries
• cargo clippy -p oar-ocr-vl --all-targets --features cuda,download-binaries -- -D warnings
• 30-page OmniDocBench v1.5 AR/DFlash CUDA benchmark described above

[GreatV]

Closed because this DFlash work was published before explicit approval. The implementation remains local for further review.

[gemini-code-assist]

Code Review

This pull request implements the DFlash parallel draft model to enable speculative decoding for HunyuanOCR 1.5, accelerating single-document inference. Key changes include adding the DFlash draft model, updating the LLM to support auxiliary intermediate outputs and KV cache trimming, and modifying causal mask generation to use integer-space comparisons to prevent BF16 precision issues in long contexts. The review feedback suggests optimizing performance in speculative decoding hot paths by creating position tensors directly on the device using Tensor::arange to avoid CPU allocation and host-to-device transfer overhead.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[gemini-code-assist]

Creating a Vec<i64> on the CPU and transferring it to the device via Tensor::from_vec introduces unnecessary CPU allocation and host-to-device transfer overhead. Since this is called in the hot path of speculative decoding, we can optimize this by creating the positions tensor directly on the device using Tensor::arange.

Suggested change

	let positions: Vec<i64> = (start..start + len).map(|x| x as i64).collect();
	let positions = Tensor::from_vec(positions, (1, 1, len), &self.device)
	.map_err(|e| tensor_err("HunyuanOCR DFlash: position tensor", e))?;
	let positions = Tensor::arange(start as i64, (start + len) as i64, &self.device)?
	.reshape((1, 1, len))
	.map_err(|e| tensor_err("HunyuanOCR DFlash: position tensor", e))?;

[gemini-code-assist]

Creating the verification positions tensor by allocating a Vec<i64> on the CPU and transferring it to the device introduces host-to-device transfer overhead in the hot loop of speculative decoding. We can optimize this by creating the positions tensor directly on the device using Tensor::arange and broadcasting it.

            let positions = Tensor::arange(context_len as i64, (context_len + num_spec + 1) as i64, &self.device)?
                .reshape((num_spec + 1, 1))?
                .broadcast_as((num_spec + 1, 4))?
                .transpose(0, 1)?
                .unsqueeze(1)
                .map_err(|e| candle_to_ocr_inference("HunyuanOCR DFlash", "verify positions", e))?;