Repository navigation
Incremental fetching / pagination in the Results Viewer #1121
clang-engineer
started this conversation in
Ideas (Feature Requests)
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
I'd like to suggest incremental fetching (or pagination / "Load More") for the Results Viewer.
Currently, when a hard fetch limit is enabled (for example, 1,000 rows), inspecting rows beyond that limit requires increasing the limit and executing the query again.
For exploratory use against large or remote databases, it would be useful if the Results Viewer could fetch additional rows on demand:
The exact implementation doesn't necessarily need to expose SQL
LIMIT/OFFSET; ideally this could continue fetching from the underlying cursor where the adapter supports it.I looked through the current implementation and noticed that
HarlequinCursorcurrently exposesfetchall(), andquery.fetch()drains the cursor into aResultSet.I also noticed that large-result streaming is already mentioned as future work requiring changes to the cursor interface. This seems closely related to #875 as well, since incremental fetching could help avoid materializing very large result sets up front.
Would incremental loading in the Results Viewer fit into that planned streaming work?
I'd be happy to work on this and submit a PR if it's something you'd be interested in supporting. If so, I'd appreciate guidance on the preferred direction for the cursor API (for example,
fetchmany()/ batch-oriented fetching) before starting the implementation.All reactions