feat: benchmark proxy payload scaling

[harivilasp]

Summary

Refs #32.

Extends the existing agentic-server proxy Criterion benchmark with Phase 1 coverage for proxy-overhead payload-size scaling:

• keeps the existing direct vs proxied non-streaming and streaming baselines
• sends explicit store: false request bodies so proxied cases exercise the proxy path
• adds non-streaming payload-size benchmarks for 1 KiB, 10 KiB, and 100 KiB prompts
• adds an integration test proving a below-limit 100 KiB store: false request reaches the mock vLLM on the proxy path with the expected request shape

The benchmark still uses the local mock LLM and local gateway, so it remains deterministic and does not require a live vLLM server.

Per review feedback, concurrency benchmarking is intentionally left out of this proxy-only benchmark and can be handled in gateway_bench.rs / database-backed gateway benchmarks.

Benchmark Results

Run locally on this branch with:

cargo +stable bench -p agentic-server --bench benches -- 'non_stream/body_size'

Prompt size	Direct median	Proxied median	Approx. overhead
1 KiB	39.900 µs	50.737 µs	+10.837 µs
10 KiB	44.493 µs	53.605 µs	+9.112 µs
100 KiB	76.618 µs	82.224 µs	+5.606 µs

These numbers use the local mock upstream and should be treated as reference data for this benchmark harness, not production latency claims.

Test Plan

• cargo +stable fmt --all --check
• rtk test cargo +stable test -p agentic-server
• cargo +stable clippy -p agentic-server --all-targets -- -D warnings
• cargo +stable test -p agentic-server --benches --no-run
• cargo +stable bench -p agentic-server --bench benches -- 'non_stream/body_size'

[franciscojavierarceo]

@harivilasp can you resolve conflicts?

[maralbahari]

@harivilasp Thanks for the PR.
I think it would be good to have concurrency benchmark be on gateway_bench.rs since all proxy_bench.rs does is to forward request to the vLLM upstream only. whereas the gateway benchmark includes conversation/responses rehydration path. We have been prioritizing sqlite support for agentic-api gateway so to truly test concurrency on gateway would require testing with postgres.

[maralbahari]

@harivilasp could you please add some benchmarking result in PR description for direct and proxied differences so that we can use for reference later on.

[harivilasp]

Sure checking

[harivilasp]

Thanks, that makes sense. This PR currently focuses on proxy payload-size overhead only and does not include a concurrency benchmark. I agree that a proper concurrency benchmark should live in gateway_bench.rs. I can keep this PR scoped to payload-size benchmarking and follow up separately with a gateway concurrency benchmark.

[ashwing]

Nice to finally have a number on whether forwarding cost scales with body size. Good timing too since the forward path is churning right now (#82 just merged, #85 is reworking gateway.rs).

One thing though: as it stands this runs once and prints numbers, and nobody's going to eyeball them against anything later. The whole reason #32 wanted this was regression detection, and that only kicks in once there's a saved baseline on main and PR branches get compared against it (criterion --save-baseline basically). That's the Phase 3 bit in the issue. I'd push to get that wiring in not long after this lands, since a regression from the gateway.rs rework is exactly the kind of thing it'd catch.

Few smaller things inline.