Coalesce cold-eye prefetch and bound foreground fetch latency - #2367
Merged
philsippl merged 1 commit intoAug 28, 2026
Merged
Conversation
A broad-matching query (prod batch 201: ~3.7k discovered candidates, 22.8s compute) exposed three compounding cold-eye weaknesses: - Discovery enqueues one tiny prefetch command per full-scan chunk; with thousands of hits the 64-deep queue dropped most commands and the rest became single-row round trips, so almost nothing was prefetched. The worker now drains and merges everything queued into one reservation pass and one batched read per database round trip, and the queue holds 4096 commands. Barriers drained mid-merge complete after that read, preserving their ordering contract. - The candidates then stalled the scan on one giant foreground `id = ANY(...)` statement (~38 KiB TOASTed rows; Postgres logged multi-second slow statements). Large fetches, foreground and prefetch alike, are now split into 512-row sub-batches issued concurrently. - That foreground read had no metric, which is why dashboards showed "DB time is short" while Postgres disagreed. It now records linear_scan_cold_foreground_db_duration/_batch_size/_records_total, and the worker records the coalesced batch size. Also raise the cold-eye LFU capacity to 12288 records (~460 MiB, three times the previous size) so recurring broad-matcher candidates stay resident between requests instead of being refetched. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XAbAnQyfsSrWH6UGcRM1FF
philsippl
force-pushed
the
codex/cpu-linear-scan-cold-prefetch
branch
from
August 27, 2026 13:53
fb6349e to
4027552
Compare
dkales
approved these changes
Aug 27, 2026
philsippl
merged commit Aug 28, 2026
7bdaac0
into
codex/cpu-linear-scan-cold-eye-cache
18 of 19 checks passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes the cold-eye tail latency exposed in prod on 2026-08-26 (batch 201, 21:50:51Z): a broad-matching uniqueness request discovered ~3.7k second-eye candidates and took 22.8 s of compute, of which ~15.5 s were the two discovered-candidate stages — almost entirely un-metered foreground Postgres reads (two server-side slow-statement alerts bracket exactly those gaps in the log timeline).
Root-cause chain and fixes:
Fetchcommand per full-scan chunk; thousands of commands overflowed the 64-deep queue (queue_skips) and the surviving ones became 1–2-row round trips, so almost nothing was actually prefetched. The prefetch worker now drains and merges everything queued into one reservation pass and one batched read per database round trip, and the queue holds 4096 commands. Barrier ordering is preserved: a barrier drained mid-merge completes after the read that covers every fetch enqueued before it.id = ANY(...)statement over ~144 MiB of TOASTed rows. Large cold fetches — foreground and prefetch alike — are now split into 512-row sub-batches issued concurrently (parallelism 4) via a shared helper.linear_scan_cold_foreground_db_duration/_batch_size/_records_total, pluslinear_scan_cold_prefetch_coalesced_recordson the worker.Additionally the cold-eye LFU capacity (default and stage/prod manifests) goes from 4096 to 12288 records (~460 MiB at ~38.4 KiB/record — well inside the deployment's memory headroom) so recurring broad-matcher candidates stay resident across requests instead of being refetched.
Validation
cargo clippy --all-targets --features db_dependent -- -D warningsiris-mpc-cpulib suite: 396 passed.cold_eye_coalesces_prefetch_commands_and_batches_large_fetches(1200 rows; 700 prefetched via ~234 small commands; single fetch across both sub-batch paths; byte-exact round-trip).e2e_linear_scan_test,e2e_linear_scan_uniqueness_test(release).Expected effect on the incident profile: the ~3.7k-candidate reads coalesce into a handful of concurrent batched statements that largely complete during the 5.6 s resident scan, and any residual foreground wait becomes visible on its own dashboard series.
Stack
Based on #2351 (cold-eye cache layer); #2348 should merge this branch after review.