[ayoub-ibm]

Description

Shifts dense-extraction quality controls from model-steering (which had hit an inflection ceiling on 4B+ models) to deterministic, structural mechanisms within the graph converter and dense orchestrator. Additionally resolves a batch of dense-pipeline recall issues through targeted orphan rescue and scalable identity reconciliation.

Key Changes

• Cardinality Control (graph_max_instances): Introduces a hard bound on the graph converter to demote discovery spam past template-declared instance caps, preventing redundant exports.
• Closed-Catalog Reference Guard: Implements an edge-filtering pass that drops references pointing to targets that were never independently extracted.
• Deterministic Co-occurrence Veto: Adds a strict sibling co-occurrence rule during alias reconciliation to prevent distinct, structurally similar entities (e.g., distinct product tiers) from being incorrectly collapsed or merged.
• Orchestrator Recall Rescue: Introduces a root-identity resolution pass alongside emission-adjacency and text-anchor rescue mechanisms in the dense orchestrator, closing key recall gaps on nested or complex documents.
• Configuration Baseline: Changes the default value of llm_input_format to auto.

Note: This PR also rolls up foundational fixes that motivated these structural changes, including orphan node rescue, optimization of reconciliation batching, and coverage-pass capping.

Type of Change

• Bug fix
• New feature
• Breaking change
• Documentation update
• Code refactoring
• Performance improvement
• Test addition or update

Checklist

• My code follows the project's style guidelines
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• I have run uv run pre-commit run --all-files and all checks pass
• I have signed off my commits with git commit -s (DCO)

Testing

• CI/CD: Full test suites green (pytest tests/unit tests/integration); ruff and mypy clean.
• Coverage: Added targeted tests for the cardinality bound, closed-catalog guard, sibling veto, orchestrator rescue passes, and benchmark aggregation.

[codecov]

Codecov Report

❌ Patch coverage is 93.98798% with 30 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...ph/core/extractors/contracts/dense/orchestrator.py	93.28%	20 Missing ⚠️
...ling_graph/core/extractors/backends/llm_backend.py	83.33%	9 Missing ⚠️
docling_graph/core/converters/graph_converter.py	99.09%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!