Open
Conversation
There was a problem hiding this comment.
Pull request overview
Refactors the JS benchmark harness to support easily parameterizing concurrency level (for planned concurrency-related driver changes) and adds new “fixed query count / variable concurrency” benchmark variants.
Changes:
- Introduces a centralized JS benchmark entrypoint (
benchmark/logic/benchmark.js) and updates existing benchmark modules to export a common function signature. - Updates benchmarker runner configs and wrapper scripts to call the centralized entrypoint (benchmark names no longer include
.js). - Adds new
*_fixedbenchmark variants in runner configs to sweep concurrency levels while keeping query count fixed.
Reviewed changes
Copilot reviewed 19 out of 19 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| benchmark/runner-config/scylladb-driver/config.yml | Switches backend commands to the centralized benchmark entrypoint; adds *_fixed variants. |
| benchmark/runner-config/cassandra-driver/config.yml | Same as above for the cassandra-driver backend; adds *_fixed variants. |
| benchmark/runner-config/run-js-benchmark.sh | Wrapper now calls benchmark/logic/benchmark.js with <driver> <benchmark-name> <N>. |
| benchmark/runner-config/run-js-benchmark-fixed.sh | New wrapper for concurrency-sweep scenarios (step controls concurrency). |
| benchmark/runner-config/config.yml | Adds benchmark definitions for *_fixed variants and adjusts default run count. |
| benchmark/logic/benchmark.js | New centralized dispatcher that loads the driver, creates a client, and invokes the selected benchmark module. |
| benchmark/logic/insert.js | Converted to exported benchmark function with default step count. |
| benchmark/logic/concurrent_insert.js | Converted to exported benchmark function; now accepts concurrencyLevel. |
| benchmark/logic/select.js | Converted to exported benchmark function delegating to parametrized_select. |
| benchmark/logic/parametrized_select.js | Refactored to accept (cassandra, client, rowCount, stepCount) rather than argv. |
| benchmark/logic/large_select.js | Converted to exported benchmark function delegating to parametrized_select. |
| benchmark/logic/paging.js | Converted to exported benchmark function with default step count. |
| benchmark/logic/concurrent_paging.js | Converted to exported benchmark function; now takes runner-provided concurrencyLevel. |
| benchmark/logic/batch.js | Converted to exported benchmark function; clarifies batch-size constant naming/comment. |
| benchmark/logic/deser.js | Converted to exported benchmark function with default step count. |
| benchmark/logic/concurrent_deser.js | Converted to exported benchmark function; now accepts concurrencyLevel. |
| benchmark/logic/ser.js | Converted to exported benchmark function with default step count. |
| benchmark/logic/concurrent_ser.js | Converted to exported benchmark function; now accepts concurrencyLevel. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This commit moves the entry points for individual benchmarks to separate files. This change will allows us to easily create new benchmarks, that are parametrized on concurrency level, without the need to create a separate copy of the files.
This adds a new version of the existing benchmarks, that try to parameterize the concurrency level, rather then the number of iterations.
225088c to
ea74b6c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs: #73
With planned changes that touch the concurrency, we would want to see the driver performance across different concurrency levels.
This PR introduces a refactor to the current benchmarks, which would allow us to add benchmarks parameterised on the concurrency level easily, and adds those benchmarks.
I would recommend reviewing the first commit in vscode (or similar), to better see what changes and what's kept unchanged
Introduces changes that were generated with LLM.
Example results [with not yet published PR] (x axis is the concurrency level on the benchmark) - I need to add beter labeling support to the benchmarking tool)