Table HTML rowspan undercount on financial documents with merged cells

[Allexzaa]

Summary

dots.mocr systematically undercounts rowspan values when extracting HTML tables
from financial documents containing multi-row merged cells. The model outputs a
plausible but incorrect rowspan value instead of the true counted value.

Environment

• Model: rednote-hilab/dots.mocr
• Serving: vLLM v0.19.0, --served-model-name model, max_tokens=24000, temperature=0
• Input: PNG image, Letter page (~176 DPI equivalent, 1500×2000px)
• Prompt: prompt_layout_all_en (layout + HTML table extraction)

Reproduction

Input: A mortgage product guide with a financial LTV matrix table. The table has:

• A first column (Property Type) that spans multiple row groups via rowspan
• Multiple loan amount tiers with 2–4 rows each
• Total of 9 data rows for the 1 Unit SFR, PUD and Condo group

Expected output:

<td rowspan="9">1 Unit SFR, PUD and Condo</td>

Actual output (both 176 DPI and 300 DPI inputs):

<td rowspan="7">1 Unit SFR, PUD and Condo</td>

Rowspan integrity failure

For a valid HTML table, the sum of all rowspan values in any column must equal
the total row count. The model's output fails this check:

Column 1 rowspan sum:  7 (1 Unit SFR) + 2 (orphaned $3M rows) + 2 (2-4 Units) = 11
Actual total rows:     9 + 2 = 11
Difference:            rowspan=7 leaves 2 rows with no Property Type cell

Resolution test (176 DPI → 340 DPI)

We tested the same image upsampled to 300, 320, 330, 340, 350, 360 DPI to isolate
whether visual patch resolution was the cause. Results:

DPI	Pixels	Cash-Out 1 Unit SFR rowspan	Purchase $2.5M rowspan
176	3.0M	7 (correct: 9)	3 (correct: 2)
300	8.4M	7 (correct: 9)	3 (correct: 2)
320	9.6M	7 (correct: 9)	3 (correct: 2)
330	10.2M	8 (correct: 9)	3 (correct: 2)
340	10.8M	8 (correct: 9)	3 (correct: 2)

Tripling the pixel count (176→340 DPI, 3× more patches per table row) did not fix
the error. We conclude this is a decoder generation issue, not a visual
perception issue. The model appears to pattern-match to a statistically plausible
rowspan value from training distribution rather than counting visible rows.

Additional finding: above 320 DPI the Purchase/Refi table structure degrades
further — phantom rows are hallucinated and row order is scrambled — suggesting
higher resolution can make structure worse for complex tables.

Positive findings

• Cell values are correct in all outputs — text content, LTV%, FICO scores are accurately extracted. Only structural metadata (rowspan values, row attribution) is wrong.
• colspan detection improves with resolution: merged header cells (colspan="2") were correctly identified at 300+ DPI but missed at 176 DPI.
• Small tables (Program Codes) detected at 320+ DPI but missed at 176/300 DPI.

Analysis

The error is consistent across 7 resolution levels with temperature=0, confirming
it is systematic and not stochastic. The wrong values (7, 8 instead of 9) are
plausible rowspan values for financial tables — consistent with a pattern-matching
failure mode rather than a perception failure.

This aligns with the known limitation stated in the model card:
"The extraction of complex tables and mathematical formulas persists as a difficult task given the model's compact architecture."

Workaround

Post-hoc rowspan integrity validation (per-column rowspan sum == total rows)
catches all such errors deterministically. We are implementing a
Docling/TableFormer fallback for flagged tables as a correction layer.

Request

1. Is this a known issue with a planned fix?
2. Are there prompt strategies or inference settings that improve rowspan accuracy on merged-cell financial tables?
3. Is fine-tuning on financial table data planned as part of the stated "targeted data scaling and task-specific training" roadmap?

[goi-2025]

Facing same issue . Did you get any workaround