Repository navigation
Conversation
A restored result cache keeps the errors, collected data and other
per-file entries of the files that are going to be re-analysed. Nothing
reads them: the merge in process() replaces them with the fresh results.
restore() now drops them.
Memory that the parent process frees stays in the Zend allocator. A
pcntl_fork()ed worker shares it copy-on-write, allocates into it and so
gets a private copy of it. Each worker does this, so the waste grows with
the number of workers. ForkedProcess now calls gc_mem_caches() before the
fork to give that memory back to the OS.
The two changes need each other. Dropping the entries alone frees more
memory that the workers then copy. gc_mem_caches() alone cannot release
the allocator chunks that the stale entries keep in use.
Whole process tree PSS, 4001 files that extend one class, 20 forked
workers, class signature changed after a cold run:
cold re-analyse all
2.3.0 2.0 GB 10.2 - 10.4 GB
2.3.0 + this change 1.9 GB 2.2 - 2.3 GB
2.2.14 2.1 GB 7.6 - 8.4 GB
2.2.14 + drop only 2.1 GB 13.0 GB
2.2.14 + gc only 2.1 GB 3.8 GB
Fixes phpstan/phpstan#15387
Co-Authored-By: Claude Code
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes phpstan/phpstan#15387
An incremental run that re-analyses a large part of the project uses about 5x the memory of a run without a result cache. The cause is the forked workers: they inherit the memory that
restore()left in the main process.ResultCacheManager::restore()now drops the cached errors, locally ignored errors, lines to ignore, unmatched line ignores, collected data and exported nodes of the files in$filesToAnalyse. Nothing reads them: everymerge*()method inprocess()replaces or unsets the entries of these files. The skip-rewrite check compares the sections only whengetFilesToAnalyse() === [], so it does not change.ForkedProcess::start()callsgc_mem_caches()beforepcntl_fork(). Without it, each worker allocates into the memory that the parent freed and gets a private copy of it.The two changes need each other. Dropping the entries alone frees more memory that the workers then copy.
gc_mem_caches()alone cannot release the allocator chunks that the stale entries keep in use.Peak sum of
Pssover the main process and the 20 forked workers, with the reproducer from the issue (4001 files, signature of their common parent class changed after a cold run):The same measurement on 2.2.14 shows that each change alone does not help:
gc_mem_caches()onlyThe result-cache e2e scenarios pass locally (52 of 54; the two others need a
.gitdirectory and a CI environment variable, and fail without this change too).— Claude
Co-Authored-By: Claude Code