-
Notifications
You must be signed in to change notification settings - Fork 1.5k
CSOT refactoring #1781
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
CSOT refactoring #1781
Conversation
Make TimeoutContext immutable.
# Conflicts: # driver-core/src/main/com/mongodb/internal/connection/OperationContext.java # driver-core/src/main/com/mongodb/internal/operation/AggregateOperation.java # driver-core/src/main/com/mongodb/internal/operation/AggregateOperationImpl.java # driver-core/src/main/com/mongodb/internal/operation/AggregateToCollectionOperation.java # driver-core/src/main/com/mongodb/internal/operation/BaseFindAndModifyOperation.java # driver-core/src/main/com/mongodb/internal/operation/ChangeStreamOperation.java # driver-core/src/main/com/mongodb/internal/operation/ClientBulkWriteOperation.java # driver-core/src/main/com/mongodb/internal/operation/CommandReadOperation.java # driver-core/src/main/com/mongodb/internal/operation/CountDocumentsOperation.java # driver-core/src/main/com/mongodb/internal/operation/CountOperation.java # driver-core/src/main/com/mongodb/internal/operation/CreateCollectionOperation.java # driver-core/src/main/com/mongodb/internal/operation/CreateIndexesOperation.java # driver-core/src/main/com/mongodb/internal/operation/CreateViewOperation.java # driver-core/src/main/com/mongodb/internal/operation/DistinctOperation.java # driver-core/src/main/com/mongodb/internal/operation/DropCollectionOperation.java # driver-core/src/main/com/mongodb/internal/operation/DropDatabaseOperation.java # driver-core/src/main/com/mongodb/internal/operation/DropIndexOperation.java # driver-core/src/main/com/mongodb/internal/operation/EstimatedDocumentCountOperation.java # driver-core/src/main/com/mongodb/internal/operation/FindOperation.java # driver-core/src/main/com/mongodb/internal/operation/ListCollectionsOperation.java # driver-core/src/main/com/mongodb/internal/operation/ListDatabasesOperation.java # driver-core/src/main/com/mongodb/internal/operation/ListIndexesOperation.java # driver-core/src/main/com/mongodb/internal/operation/ListSearchIndexesOperation.java # driver-core/src/main/com/mongodb/internal/operation/MapReduceToCollectionOperation.java # driver-core/src/main/com/mongodb/internal/operation/MapReduceWithInlineResultsOperation.java # driver-core/src/main/com/mongodb/internal/operation/MixedBulkWriteOperation.java # driver-core/src/main/com/mongodb/internal/operation/ReadOperation.java # driver-core/src/main/com/mongodb/internal/operation/ReadOperationCursor.java # driver-core/src/main/com/mongodb/internal/operation/ReadOperationMapReduceCursor.java # driver-core/src/main/com/mongodb/internal/operation/RenameCollectionOperation.java # driver-core/src/main/com/mongodb/internal/operation/WriteOperation.java # driver-core/src/test/functional/com/mongodb/ClusterFixture.java # driver-core/src/test/functional/com/mongodb/OperationFunctionalSpecification.groovy # driver-core/src/test/unit/com/mongodb/internal/operation/OperationUnitSpecification.groovy # driver-legacy/src/main/com/mongodb/LegacyMixedBulkWriteOperation.java # driver-legacy/src/main/com/mongodb/MongoClient.java # driver-reactive-streams/src/main/com/mongodb/reactivestreams/client/internal/MapReducePublisherImpl.java # driver-reactive-streams/src/main/com/mongodb/reactivestreams/client/internal/OperationExecutorImpl.java # driver-reactive-streams/src/main/com/mongodb/reactivestreams/client/internal/VoidReadOperationThenCursorReadOperation.java # driver-reactive-streams/src/main/com/mongodb/reactivestreams/client/internal/VoidWriteOperationThenCursorReadOperation.java # driver-sync/src/main/com/mongodb/client/internal/MapReduceIterableImpl.java # driver-sync/src/main/com/mongodb/client/internal/MongoClusterImpl.java
driver-core/src/main/com/mongodb/internal/connection/InternalStreamConnection.java
Show resolved
Hide resolved
driver-core/src/main/com/mongodb/internal/operation/SyncOperationHelper.java
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
driver-core/src/main/com/mongodb/internal/operation/AsyncChangeStreamBatchCursor.java
Show resolved
Hide resolved
driver-core/src/main/com/mongodb/internal/operation/AsyncCommandBatchCursor.java
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR refactors the Client Side Operation Timeout (CSOT) implementation by removing the OperationContext from binding classes and instead passing it explicitly to operation execution methods. The changes modernize the API design and improve consistency across sync and async operations.
Key changes:
- Removed
getOperationContext()methods from binding interfaces and implementations - Modified operation execution methods to accept
OperationContextas an explicit parameter - Updated connection source methods to take
OperationContextas parameter - Refactored
ClientSessionBindingto create session contexts at execution time rather than during binding creation
Reviewed Changes
Copilot reviewed 149 out of 150 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| driver-sync/src/test/unit/com/mongodb/client/internal/ClientSessionBindingSpecification.groovy | Updated test to verify binding delegates to wrapped binding with explicit OperationContext |
| driver-sync/src/test/resources/logback-test.xml | Changed log level from INFO to DEBUG for testing |
| driver-sync/src/test/functional/com/mongodb/client/AbstractClientSideOperationsTimeoutProseTest.java | Added imports and minor test method name changes |
| driver-sync/src/main/com/mongodb/client/internal/MongoClusterImpl.java | Refactored to create OperationContext with session context at execution time |
| driver-sync/src/main/com/mongodb/client/internal/MapReduceIterableImpl.java | Updated operation execution methods to accept OperationContext parameter |
| driver-sync/src/main/com/mongodb/client/internal/CryptConnection.java | Renamed timeout variable for consistency |
| driver-sync/src/main/com/mongodb/client/internal/CryptBinding.java | Removed getOperationContext() method and updated connection source methods |
| driver-sync/src/main/com/mongodb/client/internal/Crypt.java | Parameter renaming and method signature updates |
| driver-sync/src/main/com/mongodb/client/internal/CommandMarker.java | Parameter renaming from operationTimeout to timeout |
| driver-sync/src/main/com/mongodb/client/internal/CollectionInfoRetriever.java | Parameter renaming for consistency |
| driver-sync/src/main/com/mongodb/client/internal/ClientSessionBinding.java | Major refactoring to remove OperationContext from binding and create session context at execution time |
| driver-reactive-streams/src/test/unit/com/mongodb/reactivestreams/client/internal/ClientSessionBindingSpecification.groovy | Similar updates to sync version for async binding tests |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
...ync/src/test/functional/com/mongodb/client/AbstractClientSideOperationsTimeoutProseTest.java
Outdated
Show resolved
Hide resolved
...ync/src/test/functional/com/mongodb/client/AbstractClientSideOperationsTimeoutProseTest.java
Outdated
Show resolved
Hide resolved
driver-sync/src/main/com/mongodb/client/internal/ClientSessionBinding.java
Outdated
Show resolved
Hide resolved
driver-sync/src/main/com/mongodb/client/internal/ClientSessionBinding.java
Outdated
Show resolved
Hide resolved
...ctive-streams/src/main/com/mongodb/reactivestreams/client/internal/ClientSessionBinding.java
Outdated
Show resolved
Hide resolved
...ctive-streams/src/main/com/mongodb/reactivestreams/client/internal/ClientSessionBinding.java
Outdated
Show resolved
Hide resolved
driver-legacy/src/main/com/mongodb/LegacyMixedBulkWriteOperation.java
Outdated
Show resolved
Hide resolved
...ctive-streams/src/main/com/mongodb/reactivestreams/client/internal/ClientSessionBinding.java
Show resolved
Hide resolved
driver-sync/src/main/com/mongodb/client/internal/ClientSessionBinding.java
Show resolved
Hide resolved
…Runner." This reverts commit 6059fc4.
driver-core/src/main/com/mongodb/internal/operation/AsyncChangeStreamBatchCursor.java
Outdated
Show resolved
Hide resolved
…eStreamBatchCursor.java
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM!
Use secrets manager to initialize atlas connect variables. Change namespace in atlas and add data (user is read only). JAVA-5961
|
Full test build including test fixes from #1817 PR: Evergreen patch
|
|
Moving to draft as merge conflict resolution is required. |
# Conflicts: # driver-core/src/main/com/mongodb/internal/operation/AggregateOperationImpl.java # driver-core/src/main/com/mongodb/internal/operation/AggregateToCollectionOperation.java # driver-core/src/main/com/mongodb/internal/operation/ClientBulkWriteOperation.java # driver-core/src/main/com/mongodb/internal/operation/CommandReadOperation.java # driver-core/src/main/com/mongodb/internal/operation/CountDocumentsOperation.java # driver-core/src/main/com/mongodb/internal/operation/CountOperation.java # driver-core/src/main/com/mongodb/internal/operation/CreateCollectionOperation.java # driver-core/src/main/com/mongodb/internal/operation/CreateIndexesOperation.java # driver-core/src/main/com/mongodb/internal/operation/CreateViewOperation.java # driver-core/src/main/com/mongodb/internal/operation/DistinctOperation.java # driver-core/src/main/com/mongodb/internal/operation/DropCollectionOperation.java # driver-core/src/main/com/mongodb/internal/operation/DropDatabaseOperation.java # driver-core/src/main/com/mongodb/internal/operation/DropIndexOperation.java # driver-core/src/main/com/mongodb/internal/operation/EstimatedDocumentCountOperation.java # driver-core/src/main/com/mongodb/internal/operation/FindOperation.java # driver-core/src/main/com/mongodb/internal/operation/ListDatabasesOperation.java # driver-core/src/main/com/mongodb/internal/operation/ListIndexesOperation.java # driver-core/src/main/com/mongodb/internal/operation/ListSearchIndexesOperation.java # driver-core/src/main/com/mongodb/internal/operation/RenameCollectionOperation.java # driver-core/src/test/functional/com/mongodb/internal/operation/AsyncCommandBatchCursorTest.java # driver-core/src/test/unit/com/mongodb/internal/operation/AsyncChangeStreamBatchCursorSpecification.groovy # driver-core/src/test/unit/com/mongodb/internal/operation/AsyncCommandBatchCursorSpecification.groovy # driver-core/src/test/unit/com/mongodb/internal/operation/ChangeStreamBatchCursorTest.java # driver-core/src/test/unit/com/mongodb/internal/operation/CommandBatchCursorSpecification.groovy # driver-core/src/test/unit/com/mongodb/internal/operation/CommandBatchCursorTest.java # driver-legacy/src/main/com/mongodb/LegacyMixedBulkWriteOperation.java # driver-reactive-streams/src/main/com/mongodb/reactivestreams/client/internal/MapReducePublisherImpl.java # driver-reactive-streams/src/main/com/mongodb/reactivestreams/client/internal/VoidReadOperationThenCursorReadOperation.java # driver-reactive-streams/src/main/com/mongodb/reactivestreams/client/internal/VoidWriteOperationThenCursorReadOperation.java # driver-sync/src/main/com/mongodb/client/internal/MapReduceIterableImpl.java DRIVERS-3293: Add DRIVERS ticket requirement to PR checklist (#1844)
| import static com.mongodb.MongoNamespace.COMMAND_COLLECTION_NAME; | ||
| import static com.mongodb.assertions.Assertions.assertFalse; | ||
| import static com.mongodb.assertions.Assertions.assertNotNull; | ||
| import static com.mongodb.assertions.Assertions.assertTrue; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this could use MongoNamespace.ADMIN_DB_COMMAND_NAMESPACE
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agreed - updated to use ADMIN_DB_COMMAND_NAMESPACE.
However, the "admin" constant seems to be currently duplicated in multiple places of the codebase, not just in ClientBulkWriteOperation. i suggest we replace the rest with MongoNamespace.ADMIN_DB_COMMAND_NAMESPACE in a separate cleanup PR.
driver-core/src/main/com/mongodb/internal/operation/ClientBulkWriteOperation.java
Outdated
Show resolved
Hide resolved
…WriteOperation.java Co-authored-by: Nabil Hachicha <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM - lets merge this.
I don't think the prose tests are critical failures for the alpha release - more a prose test issue. Worst case is merge this into main and add a ticket to review those test cases. #1833 might be the required fix.
Ship it!
|
All of the following tests that also appear as failures on mainline: Test cause analysiscom.mongodb.reactivestreams.client.ClientSideOperationTimeoutProseTest.Should_use_connectTimeoutMS_when_establishing_connection_in_background com.mongodb.client.ClientSideOperationTimeoutProseTest.Should_use_connectTimeoutMS_when_establishing_connection_in_background com.mongodb.internal.connection.AwsAuthenticationSpecification.should_not_authenticate_when_provider_gives_invalid_session_token__async__true___0_ com.mongodb.internal.connection.AwsAuthenticationSpecification.should_not_authenticate_when_provider_gives_invalid_session_token__async__false___1_ com.mongodb.reactivestreams.client.CrudProseTest.6._MongoClient.bulkWrite_handles_individual_WriteErrors_across_batches--ordered_false |
Description
Following a CSOT code walkthrough in scope of Client Bulk Write API, we identified several improvements to make the Client-Side Operations Timeout (CSOT) code more predictable, consistent, and maintainable. This PR introduces naming refinements, clearer APIs, and safer handling of TimeoutContext to improve readability and reduce potential misuse.
The main change in this PR is that methods whose execution time can be limited by a timeout, or which are executed on a particular
ClientSessionorRequestContext, must now accept a separateOperationContextparameter.The guiding principle is to avoid storing
OperationContextin fields whenever possible. Instead, theOperationContextis passed explicitly, making it more predictable where a timeout originates and easier to trace through the call chain. SinceOperationContextpropagates down the tree of method calls, it can also be overridden at any level.Some exceptions remain in entities such as
CursorandClientSession, which must hold a reference to a specificOperationContextfor their entire lifecycle. However, when any of their methods is called, a newOperationContextis created from the original to overrode a tImeout and passed down the call chain. This makes it easier to trace overrides and insert debugging hooks in the methods where contexts are instantiated.To further improve predictability and reasoning about the code, both
OperationContextandTimeoutContextare now immutable.Key changes
OperationContextfromClusterBindingandConnectionSource, requiring callers to explicitly provide it when invoking APIs such asgetConnectionSource(OperationContext)andgetConnection(OperationContext). By making the context an explicit parameter rather than relying on scattered internal state, the API becomes more predictable, easier to reason about, less prone to subtle lifecycle bugs and easier to compare with the CSOT spec.CoreCursor, allowing both tailable and non-tailable cursor operations -whether in ITERATION or CURSOR_LIFETIME mode (as implemented byCommandBatchCursor) to wrap aCoreCursorand manage timeouts by passing anOperationContexttonext,tryNext, andclosemethods. This separation also enablesChangeStreamCursorto wrapCoreCursorand implement its distinct timeout semantics without conflicting withCommandBatchCursor’slogic. Previously,CommandBatchCursorcarried two unrelated concerns, including the resetTimeoutOnClosing flag, which existed solely to letChangeStreamCursor` override timeout behavior. The new approach enforces a more clear separation of concerns, with each cursor type independently managing its timeout policy.Diagrams to present the changes made in cursors: CSOT refactoring #1781 (comment)
SyncOperationHelperandAsyncOperationHelpernow propagateOperationContextto callbacks passed intowithSourceAndConnection. Callbacks receive both theConnectionand theOperationContext. WithinwithSourceAndConnection, theOperationContextpassed to the callback includes minRTT, as the method creates a new context after selecting theConnectionSource. This allows settingminRttwithout mutating the original context fromOperation.execute(), so in a callback a caller can chose whatOperationContextto use further in the call chain, the original one withoutminRttor the one withminRttset.withSourceAndConnectionencapsulates the server selection timeout logic, computing the effective timeout(min(serverSelectionTimeout, timeoutMS))per the specification and applying it to both server selection and connection checkout. This keeps the logic centralized and original context immutable.JAVA-5640 - CSOT clarity enhancement/refactoring.
JAVA-5644 - this issue is also fixed since
TimeoutContextis fully immutable now.