Deflake centrality-child RSS isolation proof (best-of-N per arm)#54
Open
danielhertz1999-bit wants to merge 1 commit into
Open
Conversation
test_parent_rss_lower_with_centrality_child intermittently failed on macOS CI with e.g. in_parent_delta=64.9MB vs isolated_delta=68.0MB -- a ~3MB inversion on a ~65MB measurement, i.e. allocator noise, not a real regression (the test's own skip reason already calls it allocator- calibrated). Settled-RSS noise is one-sided: transient allocations and allocator arena retention only inflate the measured resident delta, never deflate it below what an arm genuinely keeps. So sample each arm n_samples (3) times and compare the minimum delta -- the cleanest estimate of the retained footprint -- which removes the single-sample spikes that flip the comparison. Pure test change; no production code touched. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
test_parent_rss_lower_with_centrality_childflakes on macOS CI. Recent failure:That's a ~3MB inversion on a ~65MB measurement — allocator noise, not a real regression. The test already flags itself as allocator-sensitive (
skipif(platform.system() != "Darwin", reason="settled-RSS isolation proof calibrated on this host's allocator")), and it passes on the overwhelming majority of runs; a single noisy sample per arm is enough to flip theisolated_delta < in_parent_deltaassertion.Fix
Settled-RSS noise is one-sided: transient allocations and allocator arena retention only ever inflate the measured resident delta — they never make an arm report less than it genuinely keeps resident. So the minimum delta across repeated identical builds is the cleanest estimate of each arm's true retained footprint.
Sample each arm
n_samples = 3times (fresh store per rep, same deterministic corpus) and compare the minima. This removes the single-sample spikes that cause the intermittent inversion while preserving the proof's intent — the isolated arm must still genuinely retain less than the in-parent arm.Pure test change; no production code is touched. The extra builds add a small amount of runtime to this one Darwin-only test.
Verification
This test is
skipif != Darwin, so it can't run off macOS — I verified the module compiles and still collects cleanly, and rely on this PR's macOS CI run for the behavioral check. If the isolation savings turn out to be genuinely marginal atn_records=3000(rather than noise), the min-of-3 comparison will surface that deterministically instead of flaking.