feat(experimental): stateless dense bucket-overlap contingency + projection API (#219)

[Fieldnote-Echo]

What

The stateless dense bucket-overlap contingency + projection API (#219, API 1 of 2), experimental-gated.

Contingency::new(&query_codes, &doc_codes, nb) builds the full nb×nb table C[a][b] = |{coords: query∈a ∧ doc∈b}| in one O(dim) histogram pass, then exposes projections mirroring ordgraph::edge::Projection:

• top_overlap, diagonal_agreement, band_agreement(r), top_group_overlap(w), bucket_l1_distance, coarsened_counts(g), rankquant_symmetric_score
• project(&weights) for learned/custom nb×nb matrices, + a Projection enum.

This is the pairwise-evidence container ordgraph migrates to (its EdgeEvidence "this is evidence for X" semantics stay in ordgraph). Stateless — no index, no persistence, never wired into a search path.

Kernel

Scalar reference + AVX-512 (avx512f/bw/vpopcntdq) histogram with a masked 64-byte tail (live-lane masking prevents tail bucket-0 false matches), runtime is_x86_feature_detected! dispatch matching #[target_feature], portable scalar fallback. #![deny(unsafe_op_in_unsafe_fn)] honored, no new deps.

Notes

• bucket_l1_distance returns u64 (the distance-weighted sum can exceed u32 for accepted dim).
• Tests: exact ordgraph parity values, scalar==SIMD parity, constructor validation, u32-overflow regression.
• Behind experimental, absent from the default build.
• Refs Promote bucket-overlap contingency/projection math: dense (pairwise) + indexed (MultiBucketBitmap) #219. Targets 0.6; may land in 0.5.0 — maintainer decides if/when to merge.

Background profiling (why this shape, and why MultiBucketBitmap is not a retriever) lives on the experiment/multibucket-profile branch.

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[gemini-code-assist]

Code Review

This pull request introduces the Contingency struct and Projection enum under the experimental feature gate, providing a stateless dense bucket-overlap contingency table for two equal-length bucket codes with both scalar and AVX-512 SIMD implementations. The reviewer suggested several performance optimizations: pre-broadcasting bucket values into registers before the loop in the AVX-512 kernel to avoid redundant splat operations, directly computing the valid range of doc buckets in band_agreement to avoid branching, and factoring out the constant row term in rankquant_symmetric_score to reduce multiplications.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[Fieldnote-Echo]

/agentic_review

[qodo-code-review]

Code Review by Qodo

🐞 Bugs (1) 📘 Rule violations (0) 📎 Requirement gaps (0)

Context used

✅ Tickets: 🎫 Promote bucket-overlap contingency/projection math: dense (pairwise) + indexed (MultiBucketBitmap)

1. Infallible nb² allocation ✓ Resolved 🐞 Bug ☼ Reliability

Description

Contingency::new allocates the nb*nb counts table with vec![0u32; table_len], which can abort
the process on allocation failure; since nb is caller-controlled, a large nb can crash the host
process instead of returning OrdvecError. This contradicts the project’s existing practice of
using fallible reservations (try_reserve_exact) when sizes may be attacker/caller influenced.

Code

src/contingency.rs[R86-93]

+        let table_len = nb
+            .checked_mul(nb)
+            .ok_or_else(|| OrdvecError::InvalidParameter {
+                name: "nb",
+                message: "bucket count squared overflows usize".to_string(),
+            })?;
+        let mut counts = vec![0u32; table_len];
+        build_histogram(query, doc, nb, &mut counts)?;

Evidence

The constructor computes table_len = nb*nb and immediately allocates with vec![0u32; table_len],
which is infallible and can abort on OOM. Elsewhere in the repo, the code explicitly avoids `vec![0;
n] for externally influenced sizes and uses try_reserve_exact` to return a recoverable error
instead.

src/contingency.rs[64-97]
src/rank_io.rs[125-136]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`Contingency::new` uses `vec![0u32; table_len]` to allocate `nb*nb` counts. If `nb` is large, allocation failure will follow Rust’s OOM path (often abort), bypassing the method’s `Result<_, OrdvecError>` contract.

## Issue Context
This crate already documents and implements fallible allocation patterns for externally influenced sizes (e.g., in rank I/O). `Contingency::new` should behave similarly and return an `OrdvecError` instead of aborting.

## Fix Focus Areas
- src/contingency.rs[64-97]
- src/rank_io.rs[125-136]

## Suggested fix
- Replace `let mut counts = vec![0u32; table_len];` with a fallible reserve + resize pattern:
 - `let mut counts: Vec<u32> = Vec::new();`
 - `counts.try_reserve_exact(table_len).map_err(|_| OrdvecError::InvalidParameter { name: "nb", message: "counts allocation too large".to_string() })?;`
 - `counts.resize(table_len, 0u32);`
- (Optional) Add a hard upper bound on `nb` (or `table_len`) appropriate for this API to avoid pathological allocations even when memory is available.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

2. MultiBucketBitmap warning not explicit ✓ Resolved 📎 Requirement gap ⚙ Maintainability

Description

The new public-facing comments added alongside Contingency do not explicitly state that
MultiBucketBitmap is not the default single-score retrieval path nor describe its intended niche
(one query vs many docs with full tables or many projections per doc). This can lead to misuse or
incorrect defaulting by downstream callers.

Code

src/lib.rs[R90-97]

+// `Contingency` / `Projection` are the stateless dense-code contingency-table
+// surface (issue #219): the full `nb × nb` bucket-overlap table for two `&[u8]`
+// code slices, plus its named projections. This is a research/analysis
+// primitive — it is *not* a retrieval index and is never wired into a search
+// path — so it lives behind the same `experimental` gate as
+// `MultiBucketBitmap`, the bilinear-decomposition surface it complements.
+#[cfg(feature = "experimental")]
+pub use contingency::{Contingency, Projection};

Evidence

PR Compliance ID 6 requires explicit documentation that MultiBucketBitmap is not the default
single-score retrieval path and describes its niche. The newly added crate-root comments mention
MultiBucketBitmap only as a complemented surface behind experimental, but do not include the
required explicit warning and niche description.

Explicitly document MultiBucketBitmap is not the default single-score retrieval path
src/lib.rs[90-97]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Documentation does not explicitly warn that `MultiBucketBitmap` is *not* the default single-score retrieval path, and it doesn’t clearly describe the intended niche use-case.

## Issue Context
Compliance requires an explicit warning/positioning statement to prevent callers from defaulting to `MultiBucketBitmap` inappropriately.

## Fix Focus Areas
- src/lib.rs[90-97]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

3. band_agreement add overflow ✓ Resolved 🐞 Bug ≡ Correctness

Description

Contingency::band_agreement computes end = (qb + radius).min(self.buckets - 1), which can
overflow usize for large radius, causing a debug overflow panic or wrapped bounds and incorrect
totals in release builds. radius is user-controlled via the public `Projection::BandAgreement {
radius }` API and is not validated/clamped.

Code

src/contingency.rs[R172-174]

+            let start = qb.saturating_sub(radius);
+            let end = (qb + radius).min(self.buckets - 1);
+            for db in start..=end {

Evidence

The code performs unchecked usize addition when computing the inclusive end of the band; this can
overflow for large radius. The radius parameter is exposed directly through
Projection::BandAgreement with no validation, so the overflow is caller-triggerable.

src/contingency.rs[163-177]
src/contingency.rs[292-316]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`Contingency::band_agreement` uses `qb + radius` to compute the end column, which can overflow `usize` before the `.min(...)` clamp is applied.

## Issue Context
This method is reachable from the public experimental API via `Projection::BandAgreement { radius }`, so callers can pass arbitrarily large `radius`.

## Fix Focus Areas
- src/contingency.rs[166-177]

## Suggested fix
- Replace `let end = (qb + radius).min(self.buckets - 1);` with `let end = qb.saturating_add(radius).min(self.buckets - 1);`.
- (Optional) Consider clamping `radius` to `self.buckets - 1` up front to avoid redundant work.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

4. Inexact f32 docs 🐞 Bug ⚙ Maintainability

Description

Projection::score documentation claims integer-valued projections are widened to f32 “without
rounding”, but as f32 casts (e.g., for BucketL1Distance) can lose integer precision once values
exceed f32’s exact-integer range. This can mislead callers into assuming the returned f32 is an
exact integer for large dim/scores.

Code

src/contingency.rs[R340-342]

+    /// Evaluate this projection against `contingency`, returning the value as
+    /// `f32`. The returned value is the scalar summary selected by the variant —
+    /// integer-valued projections are widened to `f32` without rounding.

Evidence

The doc explicitly claims “without rounding”, while the implementation converts integer projections
via as f32 (including bucket_l1_distance() as f32), which is a potentially lossy float
conversion by design.

src/contingency.rs[339-350]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`Projection::score` docs currently state that integer-valued projections are widened to `f32` "without rounding". However, the implementation uses `as f32` casts for integer projections, which can be lossy for sufficiently large integers.

### Issue Context
This is primarily a documentation/expectations mismatch: the return type is `f32`, so callers should expect floating-point semantics. But the current wording implies exact integer preservation.

### Fix Focus Areas
- src/contingency.rs[340-350]

### Suggested change
Update the doc comment to something like:
- "integer-valued projections are converted to `f32` (may lose integer precision for large values; exact only up to 16,777,216)"
Optionally, consider offering an additional API (e.g., `score_f64()` or projection-specific integer accessors) if exact integer values are important to consumers.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

[Fieldnote-Echo]

/agentic_review

[qodo-code-review]

Code review by qodo was updated up to the latest commit f965bf5

[Fieldnote-Echo]

/agentic_review

[qodo-code-review]

Code review by qodo was updated up to the latest commit f965bf5

[Fieldnote-Echo]

/agentic_review

[qodo-code-review]

Code review by qodo was updated up to the latest commit af3aab1