[agent] Bench: bun/hosted and bun/rescan take about 2.1x as long since #472 (1169ae6, "Fix Bun/vlt bundled copies left unpatched"). Wall time is up 105–117% and CPU time 88–96%. The request count is unchanged (127).
Same-machine interleaved compare (socket-patch-bench, PR #485 suite)
| round |
base |
scenario |
base wall |
head wall |
Δ wall [95% CI] |
Δ CPU |
Δ RSS |
requests |
| daily, 15 pairs + 10 confirm |
6e7ef748 (main −24h) |
bun/hosted |
144.2 ms |
302.7 ms |
+116.6% [+103.7, +129.7] |
+96.2% |
+0.0% |
127 |
| daily, 15 pairs + 10 confirm |
6e7ef748 |
bun/rescan |
132.3 ms |
277.8 ms |
+113.2% [+101.8, +123.0] |
+90.0% |
+0.0% |
127 |
| confirm, 25 pairs + 20 |
6e7ef748 |
bun/hosted |
144.6 ms |
286.5 ms |
+105.5% [+95.7, +112.7] |
+87.5% |
|
127 |
| confirm, 25 pairs + 20 |
6e7ef748 |
bun/rescan |
130.2 ms |
278.0 ms |
+113.7% [+107.0, +117.5] |
+89.9% |
|
127 |
| bisect, 11 pairs |
cbf1f748 (parent of #472) |
bun/hosted |
142.0 ms |
295.5 ms |
+109.5% [+103.6, +117.5] |
+94.8% |
|
127 |
An A/A check on the same runner (head against a copy of itself, 3 scenarios) found no regression, so the runner was not too noisy.
Hot spot
Callgrind on bun/hosted (head) puts 62% of all instructions in socket_patch_core::vendor::bun_lock_text::is_bundled_entry, nearly all of it inside serde_json::from_str::<Value>.
rewrite_bun_lock (crates/socket-patch-core/src/patch/redirect/mod.rs, the for dep in &npm { for entry in &entries { loop) calls is_bundled_entry(entry) before it checks the spec:
if is_bundled_entry(entry)
&& (spec == target_spec
|| spec == url_spec
|| is_prior_hosted_bun_spec(&spec, &fname, &dep.artifact_url))
That parses every entry's metadata object as JSON once per patched dependency: 60 patches × 3000 entries is 180k JSON parses, where the old loop did string compares. Proposed fix (not applied):
- swap the operands so the cheap spec match runs first and
is_bundled_entry only runs on a matching entry; or
- compute
bundled once per entry before the dep loop.
The behavior stays the same either way. bundled_matches in vendor/bun_lock.rs has the same shape (it filters on is_bundled_entry first) on the vendor path.
Repro
export CARGO_PROFILE_PERF_INHERITS=release CARGO_PROFILE_PERF_LTO=thin CARGO_PROFILE_PERF_STRIP=none
git worktree add /tmp/base cbf1f748 && (cd /tmp/base && CARGO_TARGET_DIR=/tmp/tb cargo build --locked --profile perf -p socket-patch-cli)
# on a checkout of #485 merged with main:
cargo build --locked --profile perf -p socket-patch-cli -p socket-patch-bench
target/perf/socket-patch-bench compare --base /tmp/tb/perf/socket-patch --head target/perf/socket-patch -f '^bun/'
Runner: 4 vCPU (nproc = 4), Intel(R) Xeon(R) Processor @ 2.10GHz, cloud sandbox.
Generated by Claude Code
[agent] Bench:
bun/hostedandbun/rescantake about 2.1x as long since #472 (1169ae6, "Fix Bun/vlt bundled copies left unpatched"). Wall time is up 105–117% and CPU time 88–96%. The request count is unchanged (127).Same-machine interleaved
compare(socket-patch-bench, PR #485 suite)6e7ef748(main −24h)bun/hosted6e7ef748bun/rescan6e7ef748bun/hosted6e7ef748bun/rescancbf1f748(parent of #472)bun/hostedAn A/A check on the same runner (head against a copy of itself, 3 scenarios) found no regression, so the runner was not too noisy.
1169ae68(main), with the bench suite from Add ascanbenchmark suite and a CI performance gate #485 merged locally6e7ef748..1169ae68that touches Bun code, and the parent→Fix Bun/vlt bundled copies left unpatched (#469, #471) #472 A/B reproduces the full slowdown.Hot spot
Callgrind on
bun/hosted(head) puts 62% of all instructions insocket_patch_core::vendor::bun_lock_text::is_bundled_entry, nearly all of it insideserde_json::from_str::<Value>.rewrite_bun_lock(crates/socket-patch-core/src/patch/redirect/mod.rs, thefor dep in &npm { for entry in &entries {loop) callsis_bundled_entry(entry)before it checks the spec:That parses every entry's metadata object as JSON once per patched dependency: 60 patches × 3000 entries is 180k JSON parses, where the old loop did string compares. Proposed fix (not applied):
is_bundled_entryonly runs on a matching entry; orbundledonce per entry before thedeploop.The behavior stays the same either way.
bundled_matchesinvendor/bun_lock.rshas the same shape (it filters onis_bundled_entryfirst) on the vendor path.Repro
Runner: 4 vCPU (
nproc= 4), Intel(R) Xeon(R) Processor @ 2.10GHz, cloud sandbox.Generated by Claude Code