HDFS-17914. Prevent DFSInputStream from issuing 0-byte reads#8463
HDFS-17914. Prevent DFSInputStream from issuing 0-byte reads#8463lirui-apache wants to merge 1 commit intoapache:trunkfrom
Conversation
49a5702 to
55cf7e5
Compare
| while (true) { | ||
| // Stop once the slice is filled; an extra read with remaining()==0 | ||
| // can trigger wasted slow-lane I/O in SCR. | ||
| while (tmp.hasRemaining()) { |
There was a problem hiding this comment.
Thanks for your contribution first. Just one nit concern, is it only one difference that return at L1235(after PR) or return at L1236(before PR)? It will perform a zero-byte transfer in-memory and return immediately if tmp.remaining()=0, So it is OK for me before PR. welcome discussion if I miss something. Thanks again.
There was a problem hiding this comment.
Thanks for the review. I hit the issue when I was testing short circuit read. If we call BlockReaderLocal.read with tmp.remaining() == 0, it will hit the slow lane via BlockReaderLocal::readWithBounceBuffer::fillDataBuf, which will actually allocate and read into the bounce buffer. But since user buffer is filled, we cannot drain into it and the read data is thrown away and wasted.
Description of PR
Avoid empty buffer read in DFSInputStream#actualGetFromOneDataNode by terminating the loop once the buffer is filled.
How was this patch tested?
Added new test
For code changes:
LICENSE,LICENSE-binary,NOTICE-binaryfiles?AI Tooling
If an AI tool was used:
where is the name of the AI tool used.
https://www.apache.org/legal/generative-tooling.html