Skip to content

api: add optional composite SignBitmap + RankQuant two-stage index #192

@Fieldnote-Echo

Description

@Fieldnote-Echo

Context

A downstream database can own both SignBitmap and RankQuant, keep them in lockstep, and compose two-stage search manually. That is enough for OrdinalDB MVP. Longer-term, the composition is common and central enough that ordvec may want an optional composite type that owns the probe/body pair and exposes add/delete/search as one object.

This is wishlist-level and should build on the smaller primitives:

This is not the same as #170, which is about composite mixed-resolution RankQuant storage. This issue is specifically a SignBitmap probe plus RankQuant body wrapper.

Evidence

  • SignBitmap is already a compact sign probe with candidate APIs: src/sign_bitmap.rs:37, src/sign_bitmap.rs:133-210.
  • RankQuant is already the exact asymmetric rerank body with subset rerank: src/quant.rs:74, src/quant.rs:520-663.
  • The docs describe the two-stage path as bitmap/sign probe → RankQuant rerank: docs/RANK_MODES.md:35-43, docs/RANK_MODES.md:299-331.
  • Current public root surface exports primitives independently: src/lib.rs:63-67.

Proposed Shape

Sketch:

pub struct SignRankQuant {
    sign: SignBitmap,
    rank: RankQuant,
    options: TwoStageCandidatePolicy,
}

impl SignRankQuant {
    pub fn new(dim: usize, bits: u8) -> Self;
    pub fn add(&mut self, vectors: &[f32]);
    pub fn search(&self, queries: &[f32], k: usize) -> SearchResults;
    pub fn search_docset(&self, queries: &[f32], docset: DocsetView<'_>, k: usize) -> SearchResults;
    pub fn swap_remove(&mut self, idx: usize) -> usize;
}

Naming is flexible. It could also live behind an experimental or composite feature until the contract settles.

Acceptance Criteria

  • Owns SignBitmap and RankQuant and keeps lengths/dimensions in sync.
  • add mutates both indexes or neither, once fallible APIs exist.
  • search uses SignBitmap candidates followed by exact RankQuant rerank.
  • search_docset / allowlist path is supported once api: add docset-constrained SignBitmap to RankQuant two-stage search #188 lands, or explicitly deferred.
  • swap_remove updates both indexes consistently once api: add swap_remove to Bitmap and SignBitmap #186 lands.
  • Exposes accessors for len, is_empty, dim, bits, and byte-size accounting.
  • Tests compare results to manual composition and verify add/delete consistency.
  • Docs make clear this is still an embedded primitive, not a database, ID map, or persistence bundle.

Non-goals

  • No external IDs.
  • No metadata filters beyond caller-owned docsets.
  • No OrdinalDB .odb semantics.
  • No productization of RankQuantFastscan or hidden approximate scoring.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions