limit batch size in main loop

[ianmacartney]

Two optimizations:

1. Only process up to 64 items at once, to keep the transaction small and optimize for starting tasks quickly over getting them all in one go.
2. Cut down how many tombstones we're reading to only the last 10 seconds, but every minute scan from the start of the table, to ensure we don't miss anything.

[ianmacartney]

• show 0.4.2, 0.4.6, and 0.4.7 side-by-side #202
• add benchmark dashboard with static hosting and multi-pool support #200
• limit batch size in main loop #203 👈 (View in Graphite)
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

[pkg-pr-new]

Open in StackBlitz

npm i https://pkg.pr.new/get-convex/workpool/@convex-dev/workpool@203

commit: 8dedad3

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository: get-convex/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 8bed0140-dde9-43ec-8a80-777e70ffd583

📥 Commits

Reviewing files that changed from the base of the PR and between 3df8a63 and 8dedad3.

📒 Files selected for processing (1)

• src/component/loop.ts

🚧 Files skipped from review as they are similar to previous changes (1)

• src/component/loop.ts

---

📝 Walkthrough

Walkthrough

This PR refines the job-processing loop's batching strategy and cursor-recovery mechanism in src/component/loop.ts. It introduces a MAIN_BATCH_SIZE constant to cap per-iteration completion and start processing, preventing unbounded writes while preserving throughput through rapid re-firing. The cursor scan buffer is tightened by reducing CURSOR_BUFFER_SEGMENTS, narrowing how far behind the current position the loop looks for out-of-order commits. The main function now explicitly branches between recovery iterations (which scan from the beginning with zeroed cursors) and normal iterations (which use offset cursors). Instrumentation is enhanced with stage-specific and recovery-state-aware timing labels, plus a new debug log for idle transitions.

Sequence Diagram(s)

sequenceDiagram
  participant LoopMain as Loop.main
  participant Snapshot as SnapshotQuery
  participant Pending as getPending
  participant Start as handleStart
  participant Status as runStatus

  LoopMain->>Snapshot: request snapshot (isRecoveryIter ? zeroed cursors : offset cursors)
  Snapshot-->>LoopMain: snapshot results
  LoopMain->>Pending: fetch pending (cap by MAIN_BATCH_SIZE)
  Pending-->>LoopMain: pending completion/cancel/start lists
  LoopMain->>Start: process each start (console.time labeled by pending starts)
  Start-->>LoopMain: start handled
  LoopMain->>Status: patch runStatus -> scheduled/idle (console.debug)
  Status-->>LoopMain: patched

Loading

🚥 Pre-merge checks | ✅ 4

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title 'limit batch size in main loop' directly matches the primary change described in the summary: adding MAIN_BATCH_SIZE to cap per-iteration pending completion processing.
Description check	✅ Passed	The description accurately describes the two main optimizations: limiting per-iteration processing to 64 items and reducing cursor offset window (tombstone reads) to 10 seconds with full scans every minute.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch ian/optimize

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[reeceyang]

could consider reusing isRecoveryIter here