Add cold-eye cache and prefetch for CPU linear scan by philsippl · Pull Request #2351 · worldcoin/iris-mpc

philsippl · 2026-08-19T11:21:23Z

Summary

Adds the cold-eye data plane on top of the exact CPU linear-scan correctness baseline in #2347. This layer does not change matching thresholds, rotations, result semantics, or the resident-eye compute kernel.

How it works

Keeps only the configured full-scan eye resident for the process lifetime.
Keeps the most recent cold-eye records in a rolling LUC window.
Holds uncommitted mutations in a versioned FIFO overlay until the database commit is acknowledged.
Starts bounded cold-eye prefetch while the resident-eye scan is still running, so database reads overlap the first-stage scan.
Uses Moka's TinyLFU cache for older hot records such as recurring supermatchers (4,096 entries by default).
Keys cold-eye entries by exact (serial_id, version_id), invalidates them on mutation, and fails the request closed on a registry/database miss.
Emits cache hit/miss, prefetch capacity/latency, and database-miss metrics.

The prefetch list excludes candidates already consumed by the concurrent known-candidate stage before reservations are created, preventing stale reservations from accumulating across requests.

Validation

cargo check -p iris-mpc-bins --bin iris-mpc-linear-scan
cargo clippy -p iris-mpc-cpu -p iris-mpc -p iris-mpc-bins --all-targets -- -D warnings
Search/cascade tests: 6 passed on this layer.
Cold-eye cache/overlay/prefetch tests: 5 passed.
DB-backed cold-eye integration test compiles with db_dependent; the deterministic full-server GPU/CPU result comparison and real-server benchmark are documented in Fused mirror scan, packed pair kernel, and lane pipelining for the CPU linear scan #2348.

Review guide

Start with the cache and prefetch state in iris-mpc-cpu/src/execution/hawk_main/iris_worker.rs.
Review candidate scheduling and I/O overlap in iris-mpc-cpu/src/execution/hawk_main/search.rs.
Review the side-only database loading API in iris-mpc-store.
Finish with persistence acknowledgement in iris-mpc/src/server/mod.rs and the cache metrics/configuration.

Stack

Add deployable CPU linear-scan backend #2347 — correctness baseline
this PR — cold-eye cache and prefetch
Fused mirror scan, packed pair kernel, and lane pipelining for the CPU linear scan #2348 — compute and pipeline performance

Prefetch reservations - Reservations are single-use and idempotent instead of use-counted. The same record was hinted by the known-candidate loop, by every first-eye chunk that matched it (reauth targets always do), and by both orientations, while stage two fetched it once after deduplication, so each reauth leaked a Ready entry until the 4096-record cap disabled prefetch for the rest of the process. The first fetch now consumes the reservation and promotes the value into the LFU, where a second consumer finds it. - The actor releases leftover reservations once a batch has finished all of its scans (IrisWorkerPool::release_prefetched), so reservations never outlive the batch that made them, including on error paths. - A failed prefetch read is logged and counted rather than latched for the next barrier. Its reservations are released, so the foreground fetch repeats the read and is the path that fails closed on a missing exact version; latching could fail a batch for a transient read the foreground would have served, or attribute an abandoned scan's error to the next batch. The db-backed test now asserts that contract. - Count prefetch hits under their own metric instead of as LFU misses. Rolling LUC window - A non-dense startup window keeps its newest contiguous run, an empty window starts at the first serial ID it sees, and a skipped serial ID restarts the window, all with a warning/metric instead of a panic; the window is a cache in front of the exact-version overlay, LFU, and database. Startup missing-row check uses a set instead of O(n^2). Configuration - Reject disable_persistence in linear-scan mode: the cold-eye mutation overlay is only released by result persistence. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XAbAnQyfsSrWH6UGcRM1FF

dkales · 2026-08-21T09:12:50Z

+    if search_mode == HawkSearchMode::LinearScan && disable_persistence {
+        bail!(
+            "exact CPU linear-scan mode requires persistence: the database-backed eye's \
+             mutation overlay is released only when results are persisted \
+             (SMPC__DISABLE_PERSISTENCE=false)"
+        );
+    }


Is this not a blocker for a potential shadow mode?

oh great callout, @gryaele will it actually be deployed without persistence?

hm, we need to deploy it with persistence

It will run in shadow mode with hnsw db.

dkales · 2026-08-21T09:16:51Z


        let mut db_size: usize = 0;
-        let mut cold_storage: Option<(Store, usize)> = None;
+        let mut cold_storage: Option<(Store, usize, usize, usize)> = None;


nit: this could be a struct by now

A fresh Cargo.lock hash makes the release build and tests consume nearly the whole 20-minute budget; the post-job cache upload is then cancelled, so every subsequent run is cold again and fails the same way. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XAbAnQyfsSrWH6UGcRM1FF

A broad-matching query (prod batch 201: ~3.7k discovered candidates, 22.8s compute) exposed three compounding cold-eye weaknesses: - Discovery enqueues one tiny prefetch command per full-scan chunk; with thousands of hits the 64-deep queue dropped most commands and the rest became single-row round trips, so almost nothing was prefetched. The worker now drains and merges everything queued into one reservation pass and one batched read per database round trip, and the queue holds 4096 commands. Barriers drained mid-merge complete after that read, preserving their ordering contract. - The candidates then stalled the scan on one giant foreground `id = ANY(...)` statement (~38 KiB TOASTed rows; Postgres logged multi-second slow statements). Large fetches, foreground and prefetch alike, are now split into 512-row sub-batches issued concurrently. - That foreground read had no metric, which is why dashboards showed "DB time is short" while Postgres disagreed. It now records linear_scan_cold_foreground_db_duration/_batch_size/_records_total, and the worker records the coalesced batch size. Also raise the cold-eye LFU capacity to 12288 records (~460 MiB, three times the previous size) so recurring broad-matcher candidates stay resident between requests instead of being refetched.

philsippl added 2 commits August 19, 2026 12:57

Prefetch and cache the non-resident iris eye

083ee77

lfu cache for cold eye

fe53d00

This was referenced Aug 19, 2026

Add deployable CPU linear-scan backend #2347

Open

Fused mirror scan, packed pair kernel, and lane pipelining for the CPU linear scan #2348

Open

philsippl marked this pull request as ready for review August 19, 2026 11:49

philsippl requested a review from a team as a code owner August 19, 2026 11:49

philsippl and others added 5 commits August 19, 2026 15:26

Refactor rolling LUC cache to use VecDeque

5c90a7d

Merge codex/cpu-linear-scan into cold-eye cache

7fd12da

Merge codex/cpu-linear-scan into cold-eye cache

4f6a6cd

Merge codex/cpu-linear-scan into cold-eye cache

c1f6abd

dkales previously approved these changes Aug 21, 2026

View reviewed changes

philsippl dismissed dkales’s stale review via 15505f3 August 24, 2026 12:59

philsippl added 2 commits August 24, 2026 15:38

Merge codex/cpu-linear-scan into cold-eye cache

6215a5f

Merge codex/cpu-linear-scan into cold-eye cache

ad99876

philsippl mentioned this pull request Aug 27, 2026

Coalesce cold-eye prefetch and bound foreground fetch latency #2367

Merged

leonanos8 added a commit that referenced this pull request Sep 2, 2026

Merge updated cold-eye cache (#2351) into fused mirror scan (#2348)

2c2ea47

leonanos8 mentioned this pull request Sep 2, 2026

Deploy linear scan cold-eye prefetch improvements #2371

Merged

Navigation Menu

Uh oh!

Add cold-eye cache and prefetch for CPU linear scan - #2351

Add cold-eye cache and prefetch for CPU linear scan#2351
philsippl wants to merge 11 commits into
codex/cpu-linear-scanfrom
codex/cpu-linear-scan-cold-eye-cache

philsippl commented Aug 19, 2026

Uh oh!

dkales Aug 21, 2026

Uh oh!

philsippl Aug 21, 2026

Uh oh!

gryaele Aug 21, 2026 •

edited

Loading

Uh oh!

gryaele Aug 21, 2026

Uh oh!

dkales Aug 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

philsippl commented Aug 19, 2026

Summary

How it works

Validation

Review guide

Stack

Uh oh!

dkales Aug 21, 2026

Choose a reason for hiding this comment

Uh oh!

philsippl Aug 21, 2026

Choose a reason for hiding this comment

Uh oh!

gryaele Aug 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gryaele Aug 21, 2026

Choose a reason for hiding this comment

Uh oh!

dkales Aug 21, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

gryaele Aug 21, 2026 •

edited

Loading