Skip to content

HADOOP-19790. Set Minimum Required Maven Version to 3.9.11#8214

Merged
steveloughran merged 10 commits intoapache:trunkfrom
ABFSDriver:HADOOP-19790
Mar 23, 2026
Merged

HADOOP-19790. Set Minimum Required Maven Version to 3.9.11#8214
steveloughran merged 10 commits intoapache:trunkfrom
ABFSDriver:HADOOP-19790

Conversation

@anujmodi2021
Copy link
Copy Markdown
Contributor

Description of PR

Some recent updated done in certain dependencies have caused build failure on older maven versions.
For details refer to JIRA: https://issues.apache.org/jira/browse/HADOOP-19790

With the other upgrades going in 3.5.0, we propose to upgrade the minimum required maven version to the latest stable version i.e. 3.9.12

How was this patch tested?

Negative test: Build failed when attempted with maven version lesser than required.

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-enforcer-plugin:3.5.0:enforce (clean) on project hadoop-main: [ERROR] Rule 0: org.apache.maven.enforcer.rules.version.RequireMavenVersion failed with message: [ERROR] Detected Maven Version: 3.9.6 is not in the allowed range [3.9.12,).

When attempted buid after upgrading maven to 3.9.12, build was successful

AI Tooling

No AI tools used

@anujmodi2021
Copy link
Copy Markdown
Contributor Author

Hi @steveloughran @cnauroth

Here is a small patch for changing the min required maven version.
I chose 3.9.12. Let me know if you prefer any other.

Also please let me know if any additional testig is needed.

@hadoop-yetus

This comment was marked as outdated.

@pan3793
Copy link
Copy Markdown
Member

pan3793 commented Jan 29, 2026

the current dev container uses 3.9.9, and building docs guides to use maven 3.3+, please search and udpate it everywhere for consistency

3.9.9+ might be good enough if you want to minimize the change (avoid touching Dockerfiles, but only building docs)

update: I have wrong memory, it should be 3.9.11

Comment thread BUILDING.txt Outdated
$ sudo apt-get -y install maven

Note: Above command might not the required Maven version i.e.3.9.9 or later.
In that case, download and install Maven manually from https://maven.apache.org/download.cgi
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

even ubuntu 24.04 apt repo installs old maven 3.8.7, let's just replace it with manual install commands

@github-actions github-actions Bot added the Infra label Jan 29, 2026
@anujmodi2021
Copy link
Copy Markdown
Contributor Author

Thanks a lot for your review @pan3793 and pointing out missed spots in the upgrade.
I strongly agree with your suggestion to go with a version that will be as seemless as possible.

Comment thread BUILDING.txt Outdated
$ sudo apt-get -y install openjdk-17-jdk
* Maven
$ sudo apt-get -y install maven
Most package managers do not install required version of Maven i.e. 3.9.9 or later.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add maven installation commands as well ?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Taken

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe say "compatible maven version" and then where the version requirement is declared. This is just to remove a place where changes are needed the next time a maven update is mandated

Comment thread BUILDING.txt Outdated
Comment on lines +66 to +67
Most package managers do not install required version of Maven i.e. 3.9.9 or later.
Download and Install Maven manually from https://maven.apache.org/install.html
Copy link
Copy Markdown
Member

@pan3793 pan3793 Jan 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's replace this with the real commands

Suggested change
Most package managers do not install required version of Maven i.e. 3.9.9 or later.
Download and Install Maven manually from https://maven.apache.org/install.html
$ curl -L https://repo.maven.apache.org/maven2/org/apache/maven/apache-maven/3.9.11/apache-maven-3.9.11-bin.tar.gz > apache-maven-3.9.11-bin.tar.gz
$ tar -zxvf apache-maven-3.9.11-bin.tar.gz -C /opt
$ sudo ln -s /opt/apache-maven-3.9.11/bin/mvn /usr/bin

Oops, I have the wrong memory, the dev container actually uses Maven 3.9.11, not 3.9.9, sorry for the misleading. Could you update the docs to use 3.9.11?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues, thanks for checking.
I have made the changes

Comment thread BUILDING.txt Outdated
Comment on lines +71 to +76
$ sudo nano /etc/profile.d/maven.sh
$ export M2_HOME=/opt/maven
$ export MAVEN_HOME=/opt/maven
$ export PATH=${M2_HOME}/bin:${PATH}
$ sudo chmod +x /etc/profile.d/maven.sh
$ source /etc/profile.d/maven.sh
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

below setup is unnecessary if we create a soft link /usr/bin/mvn, please keep things simple

Suggested change
$ sudo nano /etc/profile.d/maven.sh
$ export M2_HOME=/opt/maven
$ export MAVEN_HOME=/opt/maven
$ export PATH=${M2_HOME}/bin:${PATH}
$ sudo chmod +x /etc/profile.d/maven.sh
$ source /etc/profile.d/maven.sh

Copy link
Copy Markdown
Member

@pan3793 pan3793 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, except for a minor comment

@anujmodi2021
Copy link
Copy Markdown
Contributor Author

LGTM, except for a minor comment

Thanks for your review @pan3793
I have taken your comment. Will wait for some more time to allow other committers to pitch in before merging this.

Copy link
Copy Markdown
Contributor

@steveloughran steveloughran left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a signature verification stage before untarring and linking to the maven binary.

Comment thread BUILDING.txt
Comment thread BUILDING.txt Outdated
$ sudo apt-get -y install openjdk-17-jdk
* Maven
$ sudo apt-get -y install maven
Most package managers do not install required version of Maven i.e. 3.9.11 or later.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Replace with

Do not as of January 2026 install an up to date version of Maven.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it

Comment thread BUILDING.txt Outdated
* Maven
$ sudo apt-get -y install maven
Most package managers do not install required version of Maven i.e. 3.9.11 or later.
Following commands download and install Maven manually
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use the following commands to download and install Maven manually

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Taken this suggestion

@hadoop-yetus

This comment was marked as outdated.

Copy link
Copy Markdown
Contributor

@steveloughran steveloughran left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

commented.

Copy link
Copy Markdown
Contributor

@cnauroth cnauroth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

HADOOP-19790 is the last issue marked as a blocker for the 3.5.0 release. @anujmodi2021 , are you able to incorporate Steve's last round of feedback?

@steveloughran
Copy link
Copy Markdown
Contributor

@cnauroth don't let my doc comments hold up the release process. we can see who ships first: 3.4.3 or 3.5.0

@anujmodi2021
Copy link
Copy Markdown
Contributor Author

Hi @steveloughran @pan3793 @cnauroth

Really sorry for keeping this waiting. I have refereshed this PR. Please review the latest iterations taking in suggestions from Steve and Cheng.

I think ./mvnw is already incorported, does that mean we don't even need manual steps to downlod maven 3.9.11 in doc?

@pan3793
Copy link
Copy Markdown
Member

pan3793 commented Feb 24, 2026

I think ./mvnw is already incorported, does that mean we don't even need manual steps to downlod maven 3.9.11 in doc?

@anujmodi2021, we can follow Steve's previous suggestion #8214 (comment)

if mvnw is listed ... Docs can just say "use an up to date version of maven; the mvnw command will download one and validate it before execution"

The final statement might be: Install an up-to-date version of Maven, or use mvnw command, which will download one and validate it before execution.

BTW, in #8257, I also update some words in BUILDING.md, you are welcome to review that.

@anujmodi2021
Copy link
Copy Markdown
Contributor Author

Thanks @pan3793 for clarification.

I have taken the suggested change. I have kept the manual commands as well in case someone does not uses mvnw.
Please let me know if this looks good now

Copy link
Copy Markdown
Contributor

@steveloughran steveloughran left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

Copy link
Copy Markdown
Contributor

@cnauroth cnauroth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@anujmodi2021
Copy link
Copy Markdown
Contributor Author

Backmerged with trun to re-trigger build.
Previous build for some reason was never ending

@hadoop-yetus
Copy link
Copy Markdown

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 55s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+0 🆗 xmllint 0m 0s xmllint was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-1 ❌ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1 💚 mvninstall 48m 38s trunk passed
+1 💚 compile 17m 27s trunk passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04
+1 💚 compile 18m 2s trunk passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1
+1 💚 mvnsite 19m 41s trunk passed
+1 💚 javadoc 9m 35s trunk passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04
+1 💚 javadoc 9m 32s trunk passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1
+1 💚 shadedclient 156m 2s branch has no errors when building and testing our client artifacts.
-0 ⚠️ patch 156m 25s Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
+1 💚 mvninstall 44m 9s the patch passed
+1 💚 compile 19m 56s the patch passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04
+1 💚 javac 19m 56s the patch passed
+1 💚 compile 18m 1s the patch passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1
+1 💚 javac 18m 1s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 mvnsite 19m 29s the patch passed
+1 💚 javadoc 9m 36s the patch passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04
+1 💚 javadoc 9m 28s the patch passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1
+1 💚 shadedclient 73m 12s patch has no errors when building and testing our client artifacts.
_ Other Tests _
-1 ❌ unit 816m 27s /patch-unit-root.txt root in the patch failed.
+1 💚 asflicense 1m 54s The patch does not generate ASF License warnings.
1133m 30s
Reason Tests
Failed junit tests hadoop.hdfs.TestRollingUpgrade
hadoop.hdfs.server.balancer.TestBalancerWithHANameNodes
Subsystem Report/Notes
Docker ClientAPI=1.53 ServerAPI=1.53 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8214/8/artifact/out/Dockerfile
GITHUB PR #8214
Optional Tests dupname asflicense codespell detsecrets mvnsite unit compile javac javadoc mvninstall shadedclient xmllint
uname Linux 288a7f21c0b7 5.15.0-164-generic #174-Ubuntu SMP Fri Nov 14 20:25:16 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 675da5e
Default Java Ubuntu-17.0.18+8-Ubuntu-124.04.1
Multi-JDK versions /usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.10+7-Ubuntu-124.04 /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.18+8-Ubuntu-124.04.1
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8214/8/testReport/
Max. process+thread count 4023 (vs. ulimit of 5500)
modules C: . U: .
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8214/8/console
versions git=2.43.0 maven=3.9.11
Powered by Apache Yetus 0.14.1 https://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link
Copy Markdown

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 19m 58s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+0 🆗 xmllint 0m 0s xmllint was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-1 ❌ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1 💚 mvninstall 47m 38s trunk passed
+1 💚 compile 18m 22s trunk passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04
+1 💚 compile 18m 2s trunk passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1
+1 💚 mvnsite 22m 28s trunk passed
+1 💚 javadoc 9m 53s trunk passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04
+1 💚 javadoc 9m 28s trunk passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1
+1 💚 shadedclient 159m 32s branch has no errors when building and testing our client artifacts.
-0 ⚠️ patch 159m 56s Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
+1 💚 mvninstall 43m 1s the patch passed
+1 💚 compile 17m 30s the patch passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04
+1 💚 javac 17m 30s the patch passed
+1 💚 compile 18m 2s the patch passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1
+1 💚 javac 18m 2s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 mvnsite 19m 36s the patch passed
+1 💚 javadoc 9m 41s the patch passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04
+1 💚 javadoc 9m 29s the patch passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1
+1 💚 shadedclient 72m 35s patch has no errors when building and testing our client artifacts.
_ Other Tests _
-1 ❌ unit 814m 41s /patch-unit-root.txt root in the patch failed.
+1 💚 asflicense 1m 50s The patch does not generate ASF License warnings.
1150m 3s
Subsystem Report/Notes
Docker ClientAPI=1.53 ServerAPI=1.53 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8214/9/artifact/out/Dockerfile
GITHUB PR #8214
Optional Tests dupname asflicense codespell detsecrets mvnsite unit compile javac javadoc mvninstall shadedclient xmllint
uname Linux 2d48f43045ce 5.15.0-164-generic #174-Ubuntu SMP Fri Nov 14 20:25:16 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 3537761
Default Java Ubuntu-17.0.18+8-Ubuntu-124.04.1
Multi-JDK versions /usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.10+7-Ubuntu-124.04 /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.18+8-Ubuntu-124.04.1
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8214/9/testReport/
Max. process+thread count 3983 (vs. ulimit of 5500)
modules C: . U: .
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8214/9/console
versions git=2.43.0 maven=3.9.11
Powered by Apache Yetus 0.14.1 https://yetus.apache.org

This message was automatically generated.

@steveloughran steveloughran changed the title HADOOP-19790. Updating Minimum Required Maven Version For Hadoop Build HADOOP-19790. Set Minimum Required Maven Version to 3.9.11 Mar 20, 2026
@anujmodi2021
Copy link
Copy Markdown
Contributor Author

@cnauroth @steveloughran @pan3793 need your help to get this unblocked. Yetus keep failing with some timeout issue.
Any leads will be helpful.
Thanks

@pan3793
Copy link
Copy Markdown
Member

pan3793 commented Mar 22, 2026

@anujmodi2021, the failure is irrelevant, I suggest merging it directly.

@steveloughran steveloughran merged commit afcf7f2 into apache:trunk Mar 23, 2026
1 of 2 checks passed
@steveloughran
Copy link
Copy Markdown
Contributor

merged it. If it is triggering problems, we can revert.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants