HDDS-9904. Add Ratis-backed streaming reads [POC] #10743
Draft
peterxcli wants to merge 2 commits into
Draft
Conversation
peterxcli
changed the title
peterxcli
force-pushed
the
ratis-stream-read-poc-squash
branch
from
b1f50e6 to
f76cd27
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
This PR is a proof-of-concept adding a new client read path that reads RATIS-replicated blocks via the Ratis DataStream read-only API, and adds server-side handling to stream ReadBlock responses efficiently (metadata + raw bytes) including a resolver to support group-independent reads for closed containers.
Changes:
- Add
RatisDataStreamBlockInputStreamand client selection logic gated byozone.client.ratis.stream.readblock.enableplus a new read window config. - Implement server-side DataStream
query(..., WritableByteChannel)handling forReadBlock, including an on-DNClosedContainerReadResolverfor closed-container reads after group removal. - Add unit/integration tests and a manually enabled benchmark to exercise and compare the new read path.
Reviewed changes
Copilot reviewed 20 out of 20 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| pom.xml | Bumps Ratis dependency version to 3.3.0-SNAPSHOT for POC support. |
| hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/read/TestRatisDataStreamReadBlockBenchmark.java | Adds a manually enabled benchmark comparing ReadBlock streaming vs Ratis DataStream reads. |
| hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/read/TestRatisDataStreamReadBlock.java | Adds integration tests for Ratis DataStream ReadBlock and closed-container read after group removal. |
| hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/keyvalue/TestKeyValueHandler.java | Updates ReadBlock streaming tests to use DataStreamObserver<ReadBlockResponse>. |
| hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/transport/server/ratis/TestClosedContainerReadResolver.java | Adds unit coverage for the closed-container read resolver behavior. |
| hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/KeyValueHandler.java | Switches ReadBlock streaming callback to deliver protobuf metadata + separate ByteBuffer payload. |
| hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/transport/server/ratis/XceiverServerRatis.java | Registers the ClosedContainerReadResolver into Ratis parameters. |
| hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/transport/server/ratis/ContainerStateMachine.java | Implements DataStream query streaming for ReadBlock and a compatibility query path that merges responses. |
| hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/transport/server/ratis/ClosedContainerReadResolver.java | Adds resolver that bypasses Raft group lookup for closed-container ReadBlock streaming. |
| hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/transport/server/GrpcXceiverService.java | Adapts ReadBlock streaming to new ReadBlockResponse (metadata + ByteBuffer data). |
| hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/interfaces/Handler.java | Updates handler ReadBlock streaming API to use DataStreamObserver<ReadBlockResponse>. |
| hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/interfaces/ContainerDispatcher.java | Introduces ReadBlockResponse wrapper and updates streamDataReadOnly signature. |
| hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/impl/HddsDispatcher.java | Updates dispatcher ReadBlock streaming to forward ReadBlockResponse and propagate failures. |
| hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/protocolPB/ContainerCommandResponseBuilders.java | Adds overload to build ReadBlock responses with a provided ByteString payload. |
| hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/DatanodeVersion.java | Adds a new DN feature version constant for Ratis DataStream ReadBlock support. |
| hadoop-hdds/client/src/test/java/org/apache/hadoop/ozone/client/io/TestBlockInputStreamFactoryImpl.java | Adds tests covering selection and compatibility fallback for the new stream type. |
| hadoop-hdds/client/src/test/java/org/apache/hadoop/hdds/scm/storage/TestRatisDataStreamBlockInputStream.java | Adds unit coverage for read window sizing logic. |
| hadoop-hdds/client/src/main/java/org/apache/hadoop/ozone/client/io/BlockInputStreamFactoryImpl.java | Adds selection of RatisDataStreamBlockInputStream when enabled and supported. |
| hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/RatisDataStreamBlockInputStream.java | Introduces the new client-side DataStream-based RATIS block input stream implementation. |
| hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/MultipartInputStream.java | Treats the new stream type as a “stream block input stream” for multipart reads. |
| hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/OzoneClientConfig.java | Adds client config toggles and validation for the Ratis DataStream read window and enable flag. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+95
to
+101
| } else if (config.isRatisStreamReadBlock() | ||
| && repConfig.getReplicationType().equals(HddsProtos.ReplicationType.RATIS) | ||
| && allDataNodesSupport(pipeline, | ||
| RATIS_DATASTREAM_READ_BLOCK_SUPPORT)) { | ||
| return new RatisDataStreamBlockInputStream(blockInfo.getBlockID(), | ||
| blockInfo.getLength(), pipeline, token, xceiverFactory, | ||
| config); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
(this is unable to compile at ci for now because the this patch use ratis 3.3.0-snapshot.)
Add Ratis-backed streaming reads
RatisDataStreamBlockInputStreamat client useDataStreamInput(returned byDataStreamApi.streamReadOnly(ByteBuffer headerMessage) DataStreamInput) to consume the data that DN write to viaStateMachine.DataApi.query(Message request, WritableByteChannel stream)ClosedContainerReadResolver, which is aDataStreamResovlerimplmentation for closed container to prevent the fake id of read pipeline cause theGroupMismatchExceptionwhen open the data stream by bypass the ratis side live group id lookup and validation.What is the link to the Apache JIRA
issues.apache.org/jira/browse/HDDS-9904
How was this patch tested?
(Please explain how this patch was tested. Ex: unit tests, manual tests, workflow run on the fork git repo.)
(If this patch involves UI changes, please attach a screenshot; otherwise, remove this.)