Skip to content

Eliminate redundant stat calls and queue shifts in getProjectFileTree - #1217

Open
nordicnode wants to merge 1 commit into
CodebuffAI:mainfrom
nordicnode:perf-project-file-tree-queue-stat
Open

Eliminate redundant stat calls and queue shifts in getProjectFileTree#1217
nordicnode wants to merge 1 commit into
CodebuffAI:mainfrom
nordicnode:perf-project-file-tree-queue-stat

Conversation

@nordicnode

Copy link
Copy Markdown

Eliminate redundant stat calls and queue shifts in getProjectFileTree

Summary

• In common/src/project-file-tree.ts, optimize getProjectFileTree to eliminate hundreds of redundant fs.stat syscalls and replace $O(N)$ queue shifts with $O(1)$ pointer indexing.
• Previously, getProjectFileTree called parseGitignore before fs.readdir(fullPath) for every directory in the crawl. Because throwOnReadError was false, parseGitignore called fileExists on .gitignore, .codebuffignore, and .manicodeignore via fs.stat — performing 3 fs.stat calls on every single directory even though fs.readdir was called immediately afterwards. In addition, directories without ignore files pushed empty ignore instances onto dirIgnores, inflating the ignore chain tested on every file.
• Reordered the traversal to call fs.readdir first, and only call parseGitignore if at least one ignore file exists in the directory. Pass the directory entries to parseGitignoreWithMode so it checks the existing set rather than calling fs.stat.
• Replaced queue.shift()! with index pointer (queue[queueIndex++]) to eliminate quadratic array reallocations during large project tree crawls.
• Benchmark: In this repository, disk fs.stat calls dropped from 2,332 down to 1,726 (606 fewer fs.stat syscalls, a 26% reduction) while generating 100% identical tree output.
• Added a unit test in common/src/__tests__/project-file-tree.test.ts asserting that directories without ignore files make zero ignore-related fs.stat calls.

Test plan

[✓] bun test src/__tests__/project-file-tree.test.ts — 13 pass, 0 fail
[✓] bun run --cwd common typecheck — 0 errors in modified files
[✓] PR hygiene check passed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant