Skip to content

Experiment with concurrency levels#425

Open
adespawn wants to merge 2 commits intoscylladb:mainfrom
adespawn:experiment-with-concurrency-levels
Open

Experiment with concurrency levels#425
adespawn wants to merge 2 commits intoscylladb:mainfrom
adespawn:experiment-with-concurrency-levels

Conversation

@adespawn
Copy link
Copy Markdown
Contributor

@adespawn adespawn commented Mar 30, 2026

Refs: #73

With planned changes that touch the concurrency, we would want to see the driver performance across different concurrency levels.

This PR introduces a refactor to the current benchmarks, which would allow us to add benchmarks parameterised on the concurrency level easily, and adds those benchmarks.

I would recommend reviewing the first commit in vscode (or similar), to better see what changes and what's kept unchanged

Introduces changes that were generated with LLM.

Example results [with not yet published PR] (x axis is the concurrency level on the benchmark) - I need to add beter labeling support to the benchmarking tool)

throughput_vs_concurrency_lelel_deser throughput_vs_concurrency_lelel

@adespawn adespawn requested a review from Copilot March 30, 2026 13:04
@adespawn adespawn self-assigned this Mar 30, 2026
@adespawn adespawn added the benchmark Things related to benchmarking the driver label Mar 30, 2026
@adespawn adespawn added this to the GA milestone Mar 30, 2026
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Refactors the JS benchmark harness to support easily parameterizing concurrency level (for planned concurrency-related driver changes) and adds new “fixed query count / variable concurrency” benchmark variants.

Changes:

  • Introduces a centralized JS benchmark entrypoint (benchmark/logic/benchmark.js) and updates existing benchmark modules to export a common function signature.
  • Updates benchmarker runner configs and wrapper scripts to call the centralized entrypoint (benchmark names no longer include .js).
  • Adds new *_fixed benchmark variants in runner configs to sweep concurrency levels while keeping query count fixed.

Reviewed changes

Copilot reviewed 19 out of 19 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
benchmark/runner-config/scylladb-driver/config.yml Switches backend commands to the centralized benchmark entrypoint; adds *_fixed variants.
benchmark/runner-config/cassandra-driver/config.yml Same as above for the cassandra-driver backend; adds *_fixed variants.
benchmark/runner-config/run-js-benchmark.sh Wrapper now calls benchmark/logic/benchmark.js with <driver> <benchmark-name> <N>.
benchmark/runner-config/run-js-benchmark-fixed.sh New wrapper for concurrency-sweep scenarios (step controls concurrency).
benchmark/runner-config/config.yml Adds benchmark definitions for *_fixed variants and adjusts default run count.
benchmark/logic/benchmark.js New centralized dispatcher that loads the driver, creates a client, and invokes the selected benchmark module.
benchmark/logic/insert.js Converted to exported benchmark function with default step count.
benchmark/logic/concurrent_insert.js Converted to exported benchmark function; now accepts concurrencyLevel.
benchmark/logic/select.js Converted to exported benchmark function delegating to parametrized_select.
benchmark/logic/parametrized_select.js Refactored to accept (cassandra, client, rowCount, stepCount) rather than argv.
benchmark/logic/large_select.js Converted to exported benchmark function delegating to parametrized_select.
benchmark/logic/paging.js Converted to exported benchmark function with default step count.
benchmark/logic/concurrent_paging.js Converted to exported benchmark function; now takes runner-provided concurrencyLevel.
benchmark/logic/batch.js Converted to exported benchmark function; clarifies batch-size constant naming/comment.
benchmark/logic/deser.js Converted to exported benchmark function with default step count.
benchmark/logic/concurrent_deser.js Converted to exported benchmark function; now accepts concurrencyLevel.
benchmark/logic/ser.js Converted to exported benchmark function with default step count.
benchmark/logic/concurrent_ser.js Converted to exported benchmark function; now accepts concurrencyLevel.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

This commit moves the entry points for individual benchmarks to separate files.
This change will allows us to easily create new benchmarks, that are parametrized
on concurrency level, without the need to create a separate copy of the files.
This adds a new version of the existing benchmarks, that try to
parameterize the concurrency level, rather then the number of iterations.
@adespawn adespawn force-pushed the experiment-with-concurrency-levels branch from 225088c to ea74b6c Compare March 30, 2026 13:27
@adespawn adespawn requested review from nikagra and wprzytula March 30, 2026 13:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

benchmark Things related to benchmarking the driver

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants