Skip to content

Deflake centrality-child RSS isolation proof (best-of-N per arm)#54

Open
danielhertz1999-bit wants to merge 1 commit into
CodeAbra:mainfrom
danielhertz1999-bit:test/deflake-centrality-rss-proof
Open

Deflake centrality-child RSS isolation proof (best-of-N per arm)#54
danielhertz1999-bit wants to merge 1 commit into
CodeAbra:mainfrom
danielhertz1999-bit:test/deflake-centrality-rss-proof

Conversation

@danielhertz1999-bit

Copy link
Copy Markdown
Contributor

Problem

test_parent_rss_lower_with_centrality_child flakes on macOS CI. Recent failure:

AssertionError: centrality child isolation did not lower the parent footprint:
in_parent_delta=64.9MB isolated_delta=68.0MB

That's a ~3MB inversion on a ~65MB measurement — allocator noise, not a real regression. The test already flags itself as allocator-sensitive (skipif(platform.system() != "Darwin", reason="settled-RSS isolation proof calibrated on this host's allocator")), and it passes on the overwhelming majority of runs; a single noisy sample per arm is enough to flip the isolated_delta < in_parent_delta assertion.

Fix

Settled-RSS noise is one-sided: transient allocations and allocator arena retention only ever inflate the measured resident delta — they never make an arm report less than it genuinely keeps resident. So the minimum delta across repeated identical builds is the cleanest estimate of each arm's true retained footprint.

Sample each arm n_samples = 3 times (fresh store per rep, same deterministic corpus) and compare the minima. This removes the single-sample spikes that cause the intermittent inversion while preserving the proof's intent — the isolated arm must still genuinely retain less than the in-parent arm.

Pure test change; no production code is touched. The extra builds add a small amount of runtime to this one Darwin-only test.

Verification

This test is skipif != Darwin, so it can't run off macOS — I verified the module compiles and still collects cleanly, and rely on this PR's macOS CI run for the behavioral check. If the isolation savings turn out to be genuinely marginal at n_records=3000 (rather than noise), the min-of-3 comparison will surface that deterministically instead of flaking.

test_parent_rss_lower_with_centrality_child intermittently failed on macOS
CI with e.g. in_parent_delta=64.9MB vs isolated_delta=68.0MB -- a ~3MB
inversion on a ~65MB measurement, i.e. allocator noise, not a real
regression (the test's own skip reason already calls it allocator-
calibrated).

Settled-RSS noise is one-sided: transient allocations and allocator arena
retention only inflate the measured resident delta, never deflate it below
what an arm genuinely keeps. So sample each arm n_samples (3) times and
compare the minimum delta -- the cleanest estimate of the retained
footprint -- which removes the single-sample spikes that flip the
comparison. Pure test change; no production code touched.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant