feat: Implement Cached and Lazy Executor Service for getFileInfo - #357
feat: Implement Cached and Lazy Executor Service for getFileInfo#357suni72 wants to merge 32 commits into
Conversation
…nject BucketPropertiesLoader to GcsFileSystemImpl
…th VisibleForTesting
…spaceStrategy interface
…ent.isHnsBucket method
…ectory and listing operations
- rename HNS configuration key to hierarchical.namespace.enable - Update test names for clarity - Reverted unrelated changes in FakeGcsFileSystemImpl Updated tests to use constants instead of hardcoded strings
…follow AAA structure
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request enhances the GCS file system client by introducing specialized executor services for improved performance and resource management. It also implements a strategy-based approach to handle different namespace models (Flat vs. Hierarchical), enabling better support for modern cloud storage features and providing a more flexible architecture for future directory operations. Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces support for hierarchical namespace (HNS) bucket resolution by adding a NamespaceStrategy interface with flat and hierarchical implementations, alongside a new LazyExecutorService to defer task execution. It also updates GcsFileSystemImpl to manage separate read and list executor services. Feedback on these changes highlights three key issues: a potential compilation error in GcsFileSystemImpl.resolveStrategy due to an unhandled checked IOException in a lambda expression, unnecessary thread pool initialization during close() when calling .get() on unused executor suppliers, and a violation of the standard Future.get(timeout, unit) contract in LazyFutureTask where synchronous execution ignores the specified timeout.
…resolution in GcsFileSystemImpl
…tion and enhance test coverage
… GcsFileSystemImpl instantiation
…gy and update list executor constants and LazyExecutorService termination behavior
39af3de to
f57dda2
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #357 +/- ##
============================================
+ Coverage 98.14% 98.23% +0.09%
- Complexity 564 597 +33
============================================
Files 38 41 +3
Lines 1779 1873 +94
Branches 167 180 +13
============================================
+ Hits 1746 1840 +94
Misses 15 15
Partials 18 18
🚀 New features to boost your workflow:
|
… prevent task execution when interrupted
suni72
changed the title
…r shutdown constant in GcsFileSystemImpl
1 participant
Type of Change
feat: A new featurefix: A bug fixdocs: Documentation only changesstyle: Changes that do not affect the meaning of the code (white-space, formatting, missing semi-colons, etc)refactor: A code change that neither fixes a bug nor adds a featureperf: A code change that improves performancetest: Adding missing tests or correcting existing testschore: Changes to the build process or auxiliary tools and libraries such as documentation generationDescription
What?
LazyExecutorServicein theclientmodule that defers task execution untilFuture.get()is invoked, running tasks synchronously on the caller's thread.analytics-core.list.parallel.enabledto control parallelized list behavior.GcsFileSystemImplto split its underlying thread pools into two dedicated executors: one for read operations (readExecutorServiceSupplier) and one for list operations (listExecutorServiceSupplier).listExecutorServiceSupplierto dynamically select its implementation: it uses a concurrent Cached Thread Pool when parallel listing is enabled, and falls back to the newly createdLazyExecutorServicewhen parallel listing is disabled.GcsClientImplTestto use centralized constants (e.g.,TEST_PROJECT,TEST_OBJECT) instead of hardcoded literal strings.LazyExecutorServiceand the modified executor initialization logic inGcsFileSystemImplTest.Why?
LazyExecutorServiceprovides a highly efficient fallback when parallel listing is disabled.Checklist
feat(core): ...)Generated/Assisted by Agent? [Yes]