feat: execute a request's invocations concurrently

[bajtos]

Written by Claude.

The server ran the invocations of one request one after another, so a request carrying 100 invocations took longer than 100 requests carrying one each, and batching made large multipart uploads slower rather than faster.

Each invocation now runs on its own goroutine, the way net/http runs each connection, admitted through a per-request semaphore the way HTTP/2 and QUIC servers cap concurrent streams per connection. server.WithMaxConcurrency(n) sets the cap. The default of 100 matches quic-go's MaxIncomingStreams and the floor RFC 9113 recommends; x/net/http2 uses 250. Zero removes the cap for deployments that bound concurrency at the listener, and a negative value panics. The semaphore is acquired before spawning, so an oversized request waits in the loop rather than as a pile of blocked goroutines.

Receipts and metadata are collected into a slice indexed by request position and merged after every invocation finishes, so the response is the same regardless of which handler finished first and needs no lock. Dispatcher errors are joined and surface after all siblings complete; they still fail the batch as before.

Handlers must be safe to call concurrently within one request, as they already had to be across requests. The HandlerFunc doc, the batch package doc, the README and notes.md say so. piri's six RPC handlers were audited before this change: their shared state is stores, mutex-guarded caches and a singleflight group, and none reads a sibling's result.

Fourth PR in the stack for FIL-1261, on top of #67, which it depends on: without the dispatcher recovering panics from handlers (#64) and from validation code (#67), a panic on one of these goroutines would crash the process.

Tests: a barrier handler that only releases once all invocations have entered, so the test deadlocks under sequential execution; cap enforcement with a cap of 2 and 5 invocations; the zero cap running 150 at once; the negative value panicking; response metadata preserved in request order when later invocations finish first. Run three times under the race detector. The examples package could not run locally because the sandbox forbids binding a TCP port; everything else in make ci passed.

Left for the dependency bump in sprue and piri: sprue's AcceptBatch comment still says the node executes sequentially, and since sprue sends up to 1000 accepts per request piri may want a higher cap.

Benchmark results

Benchmark on an M-series laptop, one request of 100 invocations:

Handler	Sequential (cap 1)	Concurrent	100 requests of one
1 ms sleep	151 ms	4.6 ms	4.5 ms
SHA-256 over 256 KiB	18.9 ms	3.7 ms	5.1 ms

One request of 1000 invocations:

case	ns/op
io/1x1000/sequential	1,562,330,862
io/1x1000	18,738,525
cpu/1x1000/sequential	187,075,104
cpu/1x1000	21,781,242

Concurrent execution is about 80x faster than sequential for the I/O handler and about 9x faster for the CPU handler.

---

_{Stack created with GitHub Stacks CLI • Give Feedback 💬}

[Copilot]

Copilot review overview

🔵 Needs a closer look

It introduces a fundamental concurrency behavior change in the server execution path, which warrants final human review despite strong test coverage.

Review effort: Lite
Findings: 2

Open (2)

• Docs misdescribe WithMaxConcurrency as a numeric limit · New
• Comment treats WithMaxConcurrency option as a limit · New

What changed in this PR

This PR changes the HTTP server’s batch execution model so invocations within a single batch request can run concurrently, controlled by a new per-request concurrency cap. This aligns batch performance more closely with sending many single-invocation requests, while keeping response assembly deterministic.

Changes:

• Execute batch invocations concurrently with a per-request semaphore and server.WithMaxConcurrency(n) (default 100; 0 disables; negative panics).
• Preserve deterministic response construction by collecting per-invocation results by request index and merging after all goroutines complete.
• Add targeted tests/benchmarks and update public documentation to reflect the new concurrency semantics and handler expectations.

File	Description
server/​options.go	Adds DefaultMaxConcurrency and WithMaxConcurrency option wired into server config.
server/​http.go	Implements concurrent ExecuteBatch with per-request semaphore and ordered merge.
server/​http_test.go	Adjusts an existing test to be safe under concurrent handler execution.
server/​concurrency_test.go	Adds concurrency behavior tests (barrier, cap enforcement, order preservation, panic on negative).
server/​bench_test.go	Adds benchmarks comparing sequential vs concurrent batching and many single requests.
README.md	Updates user-facing docs to mention concurrent batch execution and configurability.
notes.md	Records the design rationale and operational model for per-request concurrency.
execution/​execution.go	Documents handler concurrency expectations for batch execution.
execution/​batch/​batch.go	Updates package docs to reflect concurrency + ordering constraints.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4c917ee23d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-09-22T08:06:00.191665Z	4c917ee	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[alanshaw]

Can we use errgroup with SetLimit instead?

[bajtos]

I asked Claude, it rejected that idea for three reasons:

• AGENTS.md keeps the dependency list to what we have today, and golang.org/x/sync would be a new module.
• SetLimit has the opposite conventions from our option: 0 blocks every Go call and -1 means unlimited, so we would need a translation layer.
• errgroup deliberately does not propagate panics from goroutines to Wait (the comment in errgroup.go explains why), so it would not have helped with the panic boundary Codex found, which fix: recover panics from validation code #67 now handles in the dispatcher.

However, Go 1.27 added a new API that simplifies the pattern we have here, and we reworked the code to use sync.WaitGroup.Go, which removes the Add/Done boilerplate without a new dependency.

[alanshaw]

Nice 👍