HDFS-17893. Close block readers in DFSStripedInputStream on EC read exception by dlmarion · Pull Request #8348 · apache/hadoop

dlmarion · 2026-03-17T20:00:00Z

Description of PR

Replaced calls to dfsStripedInputStream.close() with calls to dfsStripedInputStream.closeCurrentBlockReaders() in StripeReader. Closing the input stream from the StripeReader causes issues with the retry functionality added in HDFS-17811. Closing the block readers may still help the issue identified in HDFS-17080.

I reduced the visibility of a couple of other methods that were not referenced anywhere else.

How was this patch tested?

Added a unit test that throws a NPE on trunk and throws an IOException in this branch.

For code changes:

Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

AI Tooling

If an AI tool was used:

The PR includes the phrase "Contains content generated by "
where is the name of the AI tool used.
My use of AI contributions follows the ASF legal policy
https://www.apache.org/legal/generative-tooling.html

Replaced calls to dfsStripedInputStream.close() with calls to dfsStripedInputStream.closeCurrentBlockReaders() in StripeReader. Closing the input stream from the StripeReader causes issues with the retry functionality added in HDFS-17811. Closing the block readers may still help the issue identified in HDFS-17080.

hadoop-yetus · 2026-03-17T22:28:48Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	14m 17s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+1 💚	mvninstall	44m 8s		trunk passed
+1 💚	compile	1m 19s		trunk passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04
+1 💚	compile	1m 20s		trunk passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1
+1 💚	checkstyle	1m 0s		trunk passed
+1 💚	mvnsite	1m 25s		trunk passed
+1 💚	javadoc	1m 11s		trunk passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04
+1 💚	javadoc	1m 10s		trunk passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1
+1 💚	spotbugs	3m 43s		trunk passed
+1 💚	shadedclient	30m 53s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 52s		the patch passed
+1 💚	compile	0m 48s		the patch passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04
+1 💚	javac	0m 48s		the patch passed
+1 💚	compile	0m 48s		the patch passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1
+1 💚	javac	0m 48s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 25s		the patch passed
+1 💚	mvnsite	0m 54s		the patch passed
+1 💚	javadoc	0m 39s		the patch passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04
+1 💚	javadoc	0m 37s		the patch passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1
+1 💚	spotbugs	3m 24s		the patch passed
+1 💚	shadedclient	29m 45s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	2m 38s		hadoop-hdfs-client in the patch passed.
+1 💚	asflicense	0m 35s		The patch does not generate ASF License warnings.
		147m 23s

Subsystem	Report/Notes
Docker	ClientAPI=1.54 ServerAPI=1.54 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8348/1/artifact/out/Dockerfile
GITHUB PR	#8348
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 55be93febf82 5.15.0-173-generic #183-Ubuntu SMP Fri Mar 6 13:29:34 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `aa5c6c1`
Default Java	Ubuntu-17.0.18+8-Ubuntu-124.04.1
Multi-JDK versions	/usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.10+7-Ubuntu-124.04 /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.18+8-Ubuntu-124.04.1
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8348/1/testReport/
Max. process+thread count	640 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs-client U: hadoop-hdfs-project/hadoop-hdfs-client
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8348/1/console
versions	git=2.43.0 maven=3.9.11 spotbugs=4.9.7
Powered by	Apache Yetus 0.14.1 https://yetus.apache.org

This message was automatically generated.

dlmarion · 2026-03-27T15:06:46Z

I added a test that causes an NPE in DFSStripedInputStream.useDirectBuffer. On trunk, the test fails with the following in the log:

[main] INFO  hdfs.TestDFSStripedInputStreamReadFailures (TestDFSStripedInputStreamReadFailures.java:testReadWithXceiverExhaustion(164)) - 7 exceptions occurred
[main] ERROR hdfs.TestDFSStripedInputStreamReadFailures (TestDFSStripedInputStreamReadFailures.java:lambda$testReadWithXceiverExhaustion$3(165)) - Exception details
java.lang.NullPointerException: Cannot invoke "org.apache.hadoop.io.erasurecode.rawcoder.RawErasureDecoder.preferDirectBuffer()" because "this.decoder" is null
        at org.apache.hadoop.hdfs.DFSStripedInputStream.useDirectBuffer(DFSStripedInputStream.java:119)
        at org.apache.hadoop.hdfs.DFSStripedInputStream.resetCurStripeBuffer(DFSStripedInputStream.java:124)
        at org.apache.hadoop.hdfs.DFSStripedInputStream.readOneStripe(DFSStripedInputStream.java:299)
        at org.apache.hadoop.hdfs.DFSStripedInputStream.readWithStrategy(DFSStripedInputStream.java:421)
        at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:960)
        at java.base/java.io.DataInputStream.read(DataInputStream.java:158)
        at org.apache.hadoop.hdfs.StripedFileTestUtil.verifyStatefulRead(StripedFileTestUtil.java:128)
        at org.apache.hadoop.hdfs.TestDFSStripedInputStreamReadFailures.lambda$testReadWithXceiverExhaustion$0(TestDFSStripedInputStreamReadFailures.java:146)
        at java.base/java.lang.Thread.run(Thread.java:1583)

The other 6 exceptions have the same stack trace. When this test is run on this branch it still fails, but in a different way which is probably handled better by a client because an IOException is expected exception. The failure in this branch looks like:

[main] INFO  hdfs.TestDFSStripedInputStreamReadFailures (TestDFSStripedInputStreamReadFailures.java:testReadWithXceiverExhaustion(164)) - 9 exceptions occurred
[main] ERROR hdfs.TestDFSStripedInputStreamReadFailures (TestDFSStripedInputStreamReadFailures.java:lambda$testReadWithXceiverExhaustion$3(165)) - Exception details
java.io.IOException: 6 missing blocks, the stripe is: AlignedStripe(Offset=0, length=1048576, fetchedChunksNum=0, missingChunksNum=6); locatedBlocks is: LocatedBlocks{;  fileLength=6291579;  underConstruction=false;  blocks=[LocatedStripedBlock{BP-1063483206-10.113.15.120-1774623121364:blk_-9223372036854775728_1005; getBlockSize()=6291579; corrupt=false; offset=0; locs=[DatanodeInfoWithStorage[127.0.0.1:38777,DS-5d4bafbc-f78a-4608-b4d7-635c26439ef6,DISK], DatanodeInfoWithStorage[127.0.0.1:33155,DS-b94250d0-39ca-47b9-ae38-14b0eccc3e2a,DISK], DatanodeInfoWithStorage[127.0.0.1:33469,DS-0ef39e40-8f52-497f-bcc4-7b74f5359200,DISK], DatanodeInfoWithStorage[127.0.0.1:40287,DS-346798b6-b6c5-4f85-ba85-84a701ced827,DISK], DatanodeInfoWithStorage[127.0.0.1:32995,DS-9be75b23-9ac5-4278-a46b-ee4a52f4ca2f,DISK], DatanodeInfoWithStorage[127.0.0.1:33789,DS-e6ff79de-2b1b-48da-a84d-a8da0fba80ff,DISK], DatanodeInfoWithStorage[127.0.0.1:46867,DS-dfa3ee1d-0964-4e1d-818d-77bdcae7c4e0,DISK], DatanodeInfoWithStorage[127.0.0.1:44495,DS-8d4dd57f-c417-4010-9278-253236ea4ff3,DISK], DatanodeInfoWithStorage[127.0.0.1:34009,DS-1af2db95-5652-4bdf-b6a8-99ef2eea080f,DISK]]; indices=[0, 1, 2, 3, 4, 5, 6, 7, 8]}];  lastLocatedBlock=LocatedStripedBlock{BP-1063483206-10.113.15.120-1774623121364:blk_-9223372036854775728_1005; getBlockSize()=6291579; corrupt=false; offset=0; locs=[DatanodeInfoWithStorage[127.0.0.1:38777,DS-5d4bafbc-f78a-4608-b4d7-635c26439ef6,DISK], DatanodeInfoWithStorage[127.0.0.1:33155,DS-b94250d0-39ca-47b9-ae38-14b0eccc3e2a,DISK], DatanodeInfoWithStorage[127.0.0.1:33469,DS-0ef39e40-8f52-497f-bcc4-7b74f5359200,DISK], DatanodeInfoWithStorage[127.0.0.1:40287,DS-346798b6-b6c5-4f85-ba85-84a701ced827,DISK], DatanodeInfoWithStorage[127.0.0.1:32995,DS-9be75b23-9ac5-4278-a46b-ee4a52f4ca2f,DISK], DatanodeInfoWithStorage[127.0.0.1:33789,DS-e6ff79de-2b1b-48da-a84d-a8da0fba80ff,DISK], DatanodeInfoWithStorage[127.0.0.1:46867,DS-dfa3ee1d-0964-4e1d-818d-77bdcae7c4e0,DISK], DatanodeInfoWithStorage[127.0.0.1:44495,DS-8d4dd57f-c417-4010-9278-253236ea4ff3,DISK], DatanodeInfoWithStorage[127.0.0.1:34009,DS-1af2db95-5652-4bdf-b6a8-99ef2eea080f,DISK]]; indices=[0, 1, 2, 3, 4, 5, 6, 7, 8]};  isLastBlockComplete=true;  ecPolicy=ErasureCodingPolicy=[Name=RS-6-3-1024k, Schema=[ECSchema=[Codec=rs, numDataUnits=6, numParityUnits=3]], CellSize=1048576, Id=1]}
        at org.apache.hadoop.hdfs.StripeReader.checkMissingBlocks(StripeReader.java:180)
        at org.apache.hadoop.hdfs.StripeReader.readStripe(StripeReader.java:350)
        at org.apache.hadoop.hdfs.DFSStripedInputStream.readOneStripe(DFSStripedInputStream.java:322)
        at org.apache.hadoop.hdfs.DFSStripedInputStream.readWithStrategy(DFSStripedInputStream.java:421)
        at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:960)
        at java.base/java.io.DataInputStream.read(DataInputStream.java:158)
        at org.apache.hadoop.hdfs.StripedFileTestUtil.verifyStatefulRead(StripedFileTestUtil.java:128)
        at org.apache.hadoop.hdfs.TestDFSStripedInputStreamReadFailures.lambda$testReadWithXceiverExhaustion$0(TestDFSStripedInputStreamReadFailures.java:146)
        at java.base/java.lang.Thread.run(Thread.java:1583)

hadoop-yetus · 2026-03-27T20:04:07Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 21s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	2m 22s		Maven dependency ordering for branch
+1 💚	mvninstall	28m 4s		trunk passed
+1 💚	compile	3m 3s		trunk passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04
+1 💚	compile	3m 16s		trunk passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1
+1 💚	checkstyle	1m 15s		trunk passed
+1 💚	mvnsite	1m 49s		trunk passed
+1 💚	javadoc	1m 22s		trunk passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04
+1 💚	javadoc	1m 26s		trunk passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1
+1 💚	spotbugs	4m 14s		trunk passed
+1 💚	shadedclient	17m 39s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 19s		Maven dependency ordering for patch
+1 💚	mvninstall	1m 20s		the patch passed
+1 💚	compile	2m 52s		the patch passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04
+1 💚	javac	2m 52s		the patch passed
+1 💚	compile	3m 7s		the patch passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1
+1 💚	javac	3m 7s		the patch passed
-1 ❌	blanks	0m 0s	/blanks-eol.txt	The patch has 10 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
-0 ⚠️	checkstyle	1m 7s	/results-checkstyle-hadoop-hdfs-project.txt	hadoop-hdfs-project: The patch generated 1 new + 15 unchanged - 0 fixed = 16 total (was 15)
+1 💚	mvnsite	1m 24s		the patch passed
+1 💚	javadoc	1m 1s		the patch passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04
+1 💚	javadoc	1m 0s		the patch passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1
+1 💚	spotbugs	4m 15s		the patch passed
+1 💚	shadedclient	17m 37s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	1m 58s		hadoop-hdfs-client in the patch passed.
-1 ❌	unit	182m 12s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
-1 ❌	asflicense	0m 25s	/results-asflicense.txt	The patch generated 1 ASF License warnings.
		283m 29s

Reason	Tests
Failed junit tests	hadoop.hdfs.TestDFSStripedInputStreamReadFailures

Subsystem	Report/Notes
Docker	ClientAPI=1.54 ServerAPI=1.54 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8348/4/artifact/out/Dockerfile
GITHUB PR	#8348
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 4df887d5b661 5.15.0-141-generic #151-Ubuntu SMP Sun May 18 21:35:19 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `2419aa2`
Default Java	Ubuntu-17.0.18+8-Ubuntu-124.04.1
Multi-JDK versions	/usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.10+7-Ubuntu-124.04 /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.18+8-Ubuntu-124.04.1
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8348/4/testReport/
Max. process+thread count	4456 (vs. ulimit of 10000)
modules	C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8348/4/console
versions	git=2.43.0 maven=3.9.11 spotbugs=4.9.7
Powered by	Apache Yetus 0.14.1 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2026-03-30T18:45:58Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 33s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	1m 58s		Maven dependency ordering for branch
+1 💚	mvninstall	42m 17s		trunk passed
+1 💚	compile	5m 25s		trunk passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04
+1 💚	compile	5m 43s		trunk passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1
+1 💚	checkstyle	2m 16s		trunk passed
+1 💚	mvnsite	3m 14s		trunk passed
+1 💚	javadoc	2m 43s		trunk passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04
+1 💚	javadoc	2m 39s		trunk passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1
+1 💚	spotbugs	7m 49s		trunk passed
+1 💚	shadedclient	30m 39s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 29s		Maven dependency ordering for patch
+1 💚	mvninstall	2m 9s		the patch passed
+1 💚	compile	4m 57s		the patch passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04
+1 💚	javac	4m 57s		the patch passed
+1 💚	compile	5m 16s		the patch passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1
+1 💚	javac	5m 16s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	1m 42s	/results-checkstyle-hadoop-hdfs-project.txt	hadoop-hdfs-project: The patch generated 1 new + 15 unchanged - 0 fixed = 16 total (was 15)
+1 💚	mvnsite	2m 25s		the patch passed
+1 💚	javadoc	1m 39s		the patch passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04
+1 💚	javadoc	1m 49s		the patch passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1
+1 💚	spotbugs	8m 39s		the patch passed
+1 💚	shadedclient	32m 46s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	2m 36s		hadoop-hdfs-client in the patch passed.
-1 ❌	unit	217m 23s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
-1 ❌	asflicense	0m 49s	/results-asflicense.txt	The patch generated 1 ASF License warnings.
		386m 2s

Reason	Tests
Failed junit tests	hadoop.hdfs.TestDFSStripedInputStreamReadFailures

Subsystem	Report/Notes
Docker	ClientAPI=1.54 ServerAPI=1.54 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8348/5/artifact/out/Dockerfile
GITHUB PR	#8348
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 492ea87d074a 5.15.0-173-generic #183-Ubuntu SMP Fri Mar 6 13:29:34 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `2ab91ac`
Default Java	Ubuntu-17.0.18+8-Ubuntu-124.04.1
Multi-JDK versions	/usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.10+7-Ubuntu-124.04 /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.18+8-Ubuntu-124.04.1
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8348/5/testReport/
Max. process+thread count	3471 (vs. ulimit of 10000)
modules	C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8348/5/console
versions	git=2.43.0 maven=3.9.11 spotbugs=4.9.7
Powered by	Apache Yetus 0.14.1 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2026-03-31T01:16:36Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 32s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	2m 20s		Maven dependency ordering for branch
+1 💚	mvninstall	42m 16s		trunk passed
+1 💚	compile	5m 21s		trunk passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04
+1 💚	compile	5m 46s		trunk passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1
+1 💚	checkstyle	2m 16s		trunk passed
+1 💚	mvnsite	3m 25s		trunk passed
+1 💚	javadoc	2m 39s		trunk passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04
+1 💚	javadoc	2m 42s		trunk passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1
+1 💚	spotbugs	7m 52s		trunk passed
+1 💚	shadedclient	30m 31s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 30s		Maven dependency ordering for patch
+1 💚	mvninstall	2m 10s		the patch passed
+1 💚	compile	4m 51s		the patch passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04
+1 💚	javac	4m 51s		the patch passed
+1 💚	compile	5m 16s		the patch passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1
+1 💚	javac	5m 16s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	1m 41s	/results-checkstyle-hadoop-hdfs-project.txt	hadoop-hdfs-project: The patch generated 1 new + 15 unchanged - 0 fixed = 16 total (was 15)
+1 💚	mvnsite	2m 20s		the patch passed
+1 💚	javadoc	1m 39s		the patch passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04
+1 💚	javadoc	1m 41s		the patch passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1
+1 💚	spotbugs	7m 11s		the patch passed
+1 💚	shadedclient	30m 4s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	2m 37s		hadoop-hdfs-client in the patch passed.
-1 ❌	unit	217m 23s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 49s		The patch does not generate ASF License warnings.
		382m 43s

Reason	Tests
Failed junit tests	hadoop.hdfs.TestDFSStripedInputStreamReadFailures

Subsystem	Report/Notes
Docker	ClientAPI=1.54 ServerAPI=1.54 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8348/6/artifact/out/Dockerfile
GITHUB PR	#8348
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 96092e5ae3f1 5.15.0-173-generic #183-Ubuntu SMP Fri Mar 6 13:29:34 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `90b3b6b`
Default Java	Ubuntu-17.0.18+8-Ubuntu-124.04.1
Multi-JDK versions	/usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.10+7-Ubuntu-124.04 /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.18+8-Ubuntu-124.04.1
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8348/6/testReport/
Max. process+thread count	3604 (vs. ulimit of 10000)
modules	C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8348/6/console
versions	git=2.43.0 maven=3.9.11 spotbugs=4.9.7
Powered by	Apache Yetus 0.14.1 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2026-04-01T00:20:38Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 34s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	2m 15s		Maven dependency ordering for branch
+1 💚	mvninstall	42m 38s		trunk passed
+1 💚	compile	5m 29s		trunk passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04
+1 💚	compile	5m 47s		trunk passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1
+1 💚	checkstyle	2m 12s		trunk passed
+1 💚	mvnsite	3m 20s		trunk passed
+1 💚	javadoc	2m 42s		trunk passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04
+1 💚	javadoc	2m 34s		trunk passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1
+1 💚	spotbugs	7m 46s		trunk passed
+1 💚	shadedclient	30m 36s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 29s		Maven dependency ordering for patch
+1 💚	mvninstall	2m 14s		the patch passed
+1 💚	compile	4m 54s		the patch passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04
+1 💚	javac	4m 54s		the patch passed
+1 💚	compile	5m 21s		the patch passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1
+1 💚	javac	5m 21s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	1m 42s	/results-checkstyle-hadoop-hdfs-project.txt	hadoop-hdfs-project: The patch generated 2 new + 15 unchanged - 0 fixed = 17 total (was 15)
+1 💚	mvnsite	2m 21s		the patch passed
+1 💚	javadoc	1m 39s		the patch passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04
+1 💚	javadoc	1m 45s		the patch passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1
+1 💚	spotbugs	7m 17s		the patch passed
+1 💚	shadedclient	30m 15s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	2m 35s		hadoop-hdfs-client in the patch passed.
+1 💚	unit	217m 30s		hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 49s		The patch does not generate ASF License warnings.
		383m 38s

Subsystem	Report/Notes
Docker	ClientAPI=1.54 ServerAPI=1.54 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8348/7/artifact/out/Dockerfile
GITHUB PR	#8348
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 393457fe5ac4 5.15.0-173-generic #183-Ubuntu SMP Fri Mar 6 13:29:34 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `7d2a019`
Default Java	Ubuntu-17.0.18+8-Ubuntu-124.04.1
Multi-JDK versions	/usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.10+7-Ubuntu-124.04 /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.18+8-Ubuntu-124.04.1
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8348/7/testReport/
Max. process+thread count	3497 (vs. ulimit of 10000)
modules	C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8348/7/console
versions	git=2.43.0 maven=3.9.11 spotbugs=4.9.7
Powered by	Apache Yetus 0.14.1 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2026-04-01T18:08:01Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 34s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	2m 4s		Maven dependency ordering for branch
+1 💚	mvninstall	42m 47s		trunk passed
+1 💚	compile	5m 19s		trunk passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04
+1 💚	compile	5m 49s		trunk passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1
+1 💚	checkstyle	2m 15s		trunk passed
+1 💚	mvnsite	3m 23s		trunk passed
+1 💚	javadoc	2m 40s		trunk passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04
+1 💚	javadoc	2m 43s		trunk passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1
+1 💚	spotbugs	7m 46s		trunk passed
+1 💚	shadedclient	30m 21s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 28s		Maven dependency ordering for patch
+1 💚	mvninstall	2m 11s		the patch passed
+1 💚	compile	4m 49s		the patch passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04
+1 💚	javac	4m 49s		the patch passed
+1 💚	compile	5m 21s		the patch passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1
+1 💚	javac	5m 21s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	1m 37s		the patch passed
+1 💚	mvnsite	2m 23s		the patch passed
+1 💚	javadoc	1m 38s		the patch passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04
+1 💚	javadoc	1m 43s		the patch passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1
+1 💚	spotbugs	7m 5s		the patch passed
+1 💚	shadedclient	30m 23s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	2m 45s		hadoop-hdfs-client in the patch passed.
+1 💚	unit	220m 2s		hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 50s		The patch does not generate ASF License warnings.
		385m 51s

Subsystem	Report/Notes
Docker	ClientAPI=1.54 ServerAPI=1.54 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8348/8/artifact/out/Dockerfile
GITHUB PR	#8348
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 515b4b3f4a9a 5.15.0-173-generic #183-Ubuntu SMP Fri Mar 6 13:29:34 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `abac061`
Default Java	Ubuntu-17.0.18+8-Ubuntu-124.04.1
Multi-JDK versions	/usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.10+7-Ubuntu-124.04 /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.18+8-Ubuntu-124.04.1
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8348/8/testReport/
Max. process+thread count	3476 (vs. ulimit of 10000)
modules	C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8348/8/console
versions	git=2.43.0 maven=3.9.11 spotbugs=4.9.7
Powered by	Apache Yetus 0.14.1 https://yetus.apache.org

This message was automatically generated.

ZanderXu · 2026-04-09T02:53:00Z

@dlmarion Thanks for your report. @hfutatzhanghb Master, can you help review this PR.

It seems that this bug is introduced by HDFS-17811, right? @dlmarion

The DFSStripedInputStream shouldn't be close in the first retry.

hfutatzhanghb · 2026-04-09T02:54:48Z

@dlmarion Thanks for your report. @hfutatzhanghb Master, can you help review this PR.

It seems that this bug is introduced by HDFS-17811, right? @dlmarion

The DFSStripedInputStream shouldn't be close in the first retry.

@ZanderXu Got, Thanks for pointing that. Will review this PR quickly.

hfutatzhanghb · 2026-04-09T06:43:31Z

Make sense to me. LGTM.

dlmarion · 2026-04-09T11:41:45Z

It seems that this bug is introduced by HDFS-17811, right? @dlmarion

The DFSStripedInputStream shouldn't be close in the first retry.

It looks like the call to dfsStripedInputStream.close() from StripeReader was added in HDFS-17080 (#5807). I mentioned HDFS-17811 because it was merged later and included changes to DFSStripedInputStream to retry the read on an IOException. I believe that this patch will allow the retry to work in more cases as an IOException is thrown in this case instead of a NullPointerException.

ZanderXu

LGTM + 1

ZanderXu · 2026-04-22T07:01:42Z

Merged. Thanks @dlmarion for your contribution and @hfutatzhanghb for your review.

github-actions Bot added HDFS trunk labels Mar 17, 2026

Added test

c4367b5

dlmarion added 2 commits March 27, 2026 15:15

Updated test comments

1384d32

Fix test

2419aa2

Removed blank lines

2ab91ac

Add missing license header

90b3b6b

Modify test to accept IOException for missing blocks

7d2a019

Fix checkstyle issues

abac061

ZanderXu approved these changes Apr 10, 2026

View reviewed changes

Merge remote-tracking branch 'upstream/trunk' into HDFS-17893-trunk

b2f54a9

ZanderXu merged commit e344f64 into apache:trunk Apr 22, 2026
1 of 2 checks passed

Conversation

dlmarion commented Mar 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description of PR

How was this patch tested?

For code changes:

AI Tooling

Uh oh!

hadoop-yetus commented Mar 17, 2026

Uh oh!

dlmarion commented Mar 27, 2026

Uh oh!

hadoop-yetus commented Mar 27, 2026

Uh oh!

hadoop-yetus commented Mar 30, 2026

Uh oh!

hadoop-yetus commented Mar 31, 2026

Uh oh!

hadoop-yetus commented Apr 1, 2026

Uh oh!

hadoop-yetus commented Apr 1, 2026

Uh oh!

ZanderXu commented Apr 9, 2026

Uh oh!

hfutatzhanghb commented Apr 9, 2026

Uh oh!

hfutatzhanghb commented Apr 9, 2026

Uh oh!

dlmarion commented Apr 9, 2026

Uh oh!

ZanderXu left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ZanderXu commented Apr 22, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

dlmarion commented Mar 17, 2026 •

edited

Loading