Benchmark: macOS packaging cache reuse

[doraemonmj]

Purpose

Personal-fork benchmark for packaging compiler identity, ccache reuse, and rejected profiling experiments. This PR exists as performance evidence and is not a merge target.

Final packaging result

Runner	Comparable baseline	Clean-head result	Difference
Ubuntu	218s	222s	+4s, effectively flat
macOS	422s	261s	-161s (-38%)

Clean-head jobs: Ubuntu, macOS.

Status

Benchmark complete and PR closed. The cleaned, single-commit change is proposed upstream in hw-native-sys/simpler#2010. Historical comments and runs remain here to document discarded alternatives.

[doraemonmj]

Final benchmark result (run 32717716227):

mode	Ubuntu	macOS
pip install .	132s	181s
targeted -> default	18s	30s
pip install -e .	15s	23s
no-build-isolation editable	12s	17s
direct CMake binding smoke	40s	52s
packaging total	217s	303s
whole job	4m24s	6m10s

Against the comparable full-Mode-5 run (Ubuntu 215s, macOS 420s), this keeps Ubuntu effectively flat (+2s) and reduces macOS by 117s. The packaging gap shrinks from 205s to 86s.

The decisive evidence was the old direct-CMake mode: Ubuntu reused 280 runtime objects and missed 29, while macOS reused only 17 and missed 291. The smoke only imports _task_interface, so rebuilding all runtimes there duplicated coverage already provided by modes 1–4. The final version builds _task_interface on both operating systems; Mode 5 now has 29 Ubuntu / 28 macOS compile misses and no redundant runtime work.

Rejected experiments are preserved in earlier workflow runs: strict CPU budgeting and forced compiler propagation both regressed at least one platform and were removed from the final commit.

[doraemonmj]

Full-coverage benchmark (commit e420aa2)\n\nBoth packaging jobs passed with all five modes intact, including Mode 5's default full CMake target.\n\n| Runner | Full-mode baseline | This change | Difference |\n| --- | ---: | ---: | ---: |\n| Ubuntu | 218s | 218s | 0s (0%) |\n| macOS | 422s | 230s | -192s (-45.5%, 1.83x faster) |\n\nThe baseline is run 32715165170, before compiler identity was unified and while Mode 5 still built the full default target. The new result is run 32724346814.\n\nThis confirms the optimization is macOS-specific in impact but platform-neutral in implementation: it restores cross-mode ccache reuse without branching on the OS, dropping modes, narrowing runtime targets, or changing smoke coverage.

[doraemonmj]

PR base cleanup: the fork's main has been fast-forwarded to upstream a7510ca, and the feature commit was republished as f273e91 with identical content. The PR now contains one commit and three files only. The benchmark from e420aa2 remains applicable because the source tree for this change is identical; CI is rerunning for the refreshed commit.

[doraemonmj]

Final personal-fork benchmark passed for the changed jobs on a40d84e:

• packaging: Ubuntu 179s; macOS 285s
• profiling: a2a3sim 155s; a5sim 156s
• profiling check critical path: 232s, down from 375-436s (38%-47%)
• coverage remains six flag combinations × two architectures, with a smoke run after every build

Tradeoff: splitting profiling removes serial latency but uses two runners. Their combined wall time is about 464s, versus 375-436s for the old single job, so this optimizes feedback latency rather than total runner consumption.

Run: https://github.com/doraemonmj/simpler_wc/actions/runs/32729558657

[doraemonmj]

Single-job simulator-context reuse benchmark

Tested reusing the first libcpu_sim_context.so artifact for the remaining 11 profiling builds. The experiment kept one runner, serial architecture scheduling, all six flag combinations, and all 12 smoke tests.

Variant	Profiling step samples	Median
reuse simulator context	306s, 320s, 325s	320s
same-period control	312s, 319s	315.5s

Reuse was 4.5s (1.4%) slower at the median. The distributions overlap and the difference is far below normal hosted-runner variance, so this optimization has no measurable benefit.

The reuse implementation was removed in 8cd5249f. The branch keeps only the personal-fork job isolation needed for faster benchmark iterations; profiling coverage is unchanged.

[doraemonmj]

Benchmarking is complete. The branch has been reduced to the validated macOS packaging compiler-identity change and will continue as a formal upstream PR. Closing this personal-fork PR avoids duplicate CI; its benchmark runs and discussion remain as supporting evidence.