Eliminate redundant stat calls and queue shifts in getProjectFileTree - #1217
Open
nordicnode wants to merge 1 commit into
Open
Eliminate redundant stat calls and queue shifts in getProjectFileTree#1217nordicnode wants to merge 1 commit into
nordicnode wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Eliminate redundant stat calls and queue shifts in getProjectFileTree
Summary
• In$O(N)$ queue shifts with $O(1)$ pointer indexing.
common/src/project-file-tree.ts, optimizegetProjectFileTreeto eliminate hundreds of redundantfs.statsyscalls and replace• Previously,
getProjectFileTreecalledparseGitignorebeforefs.readdir(fullPath)for every directory in the crawl. BecausethrowOnReadErrorwas false,parseGitignorecalledfileExistson.gitignore,.codebuffignore, and.manicodeignoreviafs.stat— performing 3fs.statcalls on every single directory even thoughfs.readdirwas called immediately afterwards. In addition, directories without ignore files pushed emptyignoreinstances ontodirIgnores, inflating the ignore chain tested on every file.• Reordered the traversal to call
fs.readdirfirst, and only callparseGitignoreif at least one ignore file exists in the directory. Pass the directory entries toparseGitignoreWithModeso it checks the existing set rather than callingfs.stat.• Replaced
queue.shift()!with index pointer (queue[queueIndex++]) to eliminate quadratic array reallocations during large project tree crawls.• Benchmark: In this repository, disk
fs.statcalls dropped from 2,332 down to 1,726 (606 fewerfs.statsyscalls, a 26% reduction) while generating 100% identical tree output.• Added a unit test in
common/src/__tests__/project-file-tree.test.tsasserting that directories without ignore files make zero ignore-relatedfs.statcalls.Test plan
[✓]
bun test src/__tests__/project-file-tree.test.ts— 13 pass, 0 fail[✓]
bun run --cwd common typecheck— 0 errors in modified files[✓] PR hygiene check passed