Repository navigation
feat: add StatementIterator#nextBatch() and #restart() - #57
Merged
Merged
Conversation
it.nextBatch(maxRows) steps up to maxRows rows in one native call. In raw mode it returns one flat array of rowCount * columnCount values; in the other modes, an array of rows. A batch with fewer than maxRows rows is the last one, and the iterator is then finished as when next() returns done. Everything else is the existing iterator's: locking, verbose logging, error cleanup and return().
it.restart(...params) runs the statement again with new bind parameters on the same iterator object, which must be finished. It skips creating and wrapping a new StatementIterator, which profiling showed to be most of the per-fetch cost of iterate() + nextBatch() for short fetches.
There was a problem hiding this comment.
🟡 Changes recommended
Legacy Node builds may fail to compile, and restart typing and documentation need updates.
2 open issues
What changed in this PR
Adds batched reads and reusable restarts to StatementIterator, with native implementation, typings, documentation, and tests.
Changes:
- Adds
nextBatch(maxRows)andrestart(...params). - Updates native iterator state and behavior.
- Adds comprehensive tests and API declarations/documentation.
| File | Summary |
|---|---|
test/29.statement.iterate.next-batch.js |
Tests batching, errors, locking, and restart behavior. |
src/objects/statement-iterator.hpp |
Adds iterator methods and execution state. |
src/objects/statement-iterator.cpp |
Implements batch stepping and restart logic. |
lib/index.d.ts |
Exposes the new iterator APIs. |
docs/api.md |
Documents nextBatch() and related iterator usage. |
Review effort: Lite
| } | ||
| UseIsolate; | ||
| UseContext; | ||
| v8::LocalVector<v8::Value> values(isolate); |
|
|
||
| Steps up to `maxRows` rows in one call, instead of one call per row as `next()` does. In [raw mode](#rawtogglestate---this), it returns one flat array of `rowCount * columnCount` values, in row order. In any other mode, it returns an array of rows, as [`.all()`](#allbindparameters---array-of-rows) would. | ||
|
|
||
| A batch with fewer than `maxRows` rows is the last one: the iterator is then finished, exactly as when `next()` returns `done`. `nextBatch()` and `next()` can be mixed, and `return()` closes the iterator early. |
This was referenced Oct 19, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.


Adds two methods to the existing
StatementIteratorso a caller can read many rows per native call, and reuse an iterator across runs. It replaces #52 (stepIntoStrings) with a much smaller API that measures the same.API
nextBatch(maxRows):maxRowsrows in one native call.rowCount * columnCountvalues, honoringsafeIntegers. In the other modes it returns an array of rows, likeall(); object rows use the cached column names from perf: cache column names per statement for object rows #53.maxRowsrows is the last one, and the iterator is then finished as whennext()returnsdone.restart(...params):StatementIterator, which profiling showed was most of the per-fetch cost for short fetches.busy, writes blocked while it's open, verbose logging once per run, error cleanup, andreturn().Why
zero-cache's TableSource reads every fetch through
iterate(), one native call per row. On a zbugs-like replica, comparing main against this API wired into zqlite's TableSource and against #52:src/against feat: add Statement#stepIntoStrings() to read rows in batches #52's +267/−6.stepIntoStrings)On a 92-row range fetch with 6 columns, building plain row objects from
nextBatch(128)costs about 245 ns per row, against about 540 ns per row throughiterate().Testing
test/29.statement.iterate.next-batch.js(14 tests) covers:nextBatch: raw values againstraw().all()for batch sizes from 1 to above the row count;safeIntegers; object, pluck and expand modes; mixing withnext()andreturn(); locking and write blocking; bind parameters cleared at the end;maxRowsvalidation; SQLite and user-defined-function errors mid-batch; verbose logging.restart(): re-running with new parameters; refusing an iterator that's still open; bad parameters leaving it finished; picking up mode changes; logging once per run.The full suite passes locally on Node 22, 24 and 26 (366 tests).