[Changming Sun (snnn)]

This pull request introduces a significant refactoring of the Azure Pipelines CI/CD infrastructure. The primary goals of these changes are to:

1. Solve the problem that vcpkg/cmake version can get changed when the CI build machine image changes, which can make pipeline suddenly broken and interrupt our release process.
2. Reduce the Zip-Nuget-Java-Nodejs Packaging Pipeline's running time by changing how macOS universal2 binaries are built.
3. Add the support for RC releases for Java packages.

Key Changes:

1. Standardized Build Tool Setup (setup-build-tools.yml)

A new reusable template, setup-build-tools.yml, has been created to centralize the setup of essential build tools.

• Pinned Dependencies: This new template allows us to pin specific versions of cmake and vcpkg, which was not previously possible. This ensures a more stable and predictable build environment across all pipelines.
• Reduced Redundancy: By consolidating the setup logic into a single template, we have significantly reduced code duplication across numerous pipeline files, making them cleaner and easier to maintain.

Currently this file is only used in macOS and Windows pipelines since most Linux pipelines use docker to manage their environment.

2. Reworked macOS Universal Binary Build Process

The methodology for building macOS universal2 binaries has been fundamentally changed to improve reliability and flexibility.

• Python Packaging: The Python packaging pipeline will no longer produce universal2 wheels. Instead, it will generate separate wheels for x86_64 and arm64 architectures.
• NuGet C-API Packaging: The NuGet C-API pipeline has been updated to first build the x86_64 and arm64 binaries independently. These single-architecture binaries are then combined to create the final universal package, rather than building a universal2 package in a single pass.

The change is made mainly because:

• Building for both ARCHs in a single pass is too slow, which may take about 5 hours in the ADO machine pool.
• A lot of MLAS features are disabled when ORT is built in such a way.

3. Java Packaging and Testing Overhaul

The Download_Java_Tools stage in "Zip-Nuget-Java-Nodejs Packaging Pipeline" is deleted because it is no longer used. Previously it was added to reduce the times of downloading the same java packages again and again , which was to reduce download errors. Now we have setup a private ADO feed for this.

Besides, there are some major changes to the pipeline that:

1. MD5 and SHA1 checksum files are provided along with the java package files instead of SHA256. This is because Sonatype's official docs says MD5/SHA1 checkcums are required while the others are optional. See: https://central.sonatype.org/publish/requirements/#supply-javadoc-and-sources . Now the publishing would fail if we don't have the MD5/SHA1 checksum files.
2. The format of the checksum files is changed. Previously we used Linux's sha256sum command to generate such files, so each checksum file contains a hash value and a filename in the file content. However, it was not the expected format. Sonatype expects that the file only contains a hash value. This PR fixes the issue.
3. A few powershell scripts were rewritten in python to improve error check and robustness
4. Added the support for generating RC packages. Previously we had to manually modify the version numbers and manually do GPG sign.
5. Two new files jar-packaging.yml and setup-maven.yml were added. We will use maven to fetch dependent packages(instead of directly HTTP fetching) to improve supply chain security, because maven allows us using a private feed to do so.

4. Dockerfile Enhancements

The Dockerfiles used in the CI have been updated to use a BASEIMAGE argument. This makes them more flexible, allowing the base image to be specified at build time, which simplifies maintenance and updates. It will allow us to using different base image repos in different CI environments. In the future we will change the Github Actions to only fetch base images from public docker repos. Meanwhile, ADO packaging pipelines will continue to use private repos.

5. Improved Release Management

The run_packaging_pipelines.py script has been updated to provide more robust and explicit control over
the package versioning for different build scenarios. This clarifies the process for generating nightly,
release candidate (RC), and final release packages.

The script now handles three distinct cases for package versioning:

• Nightly Packages: For regular CI builds (e.g., on the main branch), the script triggers the packaging
  pipelines in "nightly" mode. This sets the IsReleaseBuild parameter to false and results in packages with
  a development version suffix (e.g., 1.2.3-dev-20250909-abcdef).

• Release Candidate (RC) Builds: To create a pre-release or RC build, the script is run with the
  --build-mode release flag, along with the --pre-release-suffix-string (e.g., rc) and
  --pre-release-suffix-number (e.g., 1) arguments. This sets the IsReleaseBuild parameter to true and
  passes the suffix information to the pipelines, resulting in a semantically versioned pre-release package
  (e.g., 1.2.3-rc.1).

• Final Release Builds: For a final release, the script is run with --build-mode release without any
  pre-release suffix arguments. This sets IsReleaseBuild to true, and the resulting package will have a
  clean, final version number (e.g., 1.2.3) based on the VERSION_NUMBER file.

Please note:

• Java packages still only support the second and third mode.
• Python packages only support the first and the last mode.

To fix the vulnerability, we should sanitize the paths of all tarfile members before extraction. Specifically, before extracting each file, check that its member path does not contain "..", is not absolute, and does not escape the intended extraction directory. This requires iterating through tarfile members, validating each one, and then extracting them individually.

The recommended fix is:

• In the block starting with with tarfile.open(tgz) as tar:, replace tar.extractall(path=".") with a loop that iterates over all members, validates each member's path, and extracts only safe entries using tar.extract.
• To validate, use os.path.isabs(member.name) or ".." in pathlib.Path(member.name).parts (to check for path traversal), and optionally check that the resolved final path remains inside the extraction directory.
• Make minimal changes to the function in question; don't alter current logic elsewhere.
• Use only imports already present (i.e., use os, pathlib).

This change will only affect the extraction logic in the shown file tools/ci_build/prepare_macos_package.py, specifically in the main() function.

---

[github-advanced-security]

lintrunner found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.

To address the clear-text storage of sensitive information, we should ensure that the temporary file passphrase.txt containing the GPG passphrase is created with restrictive permissions (owner read/write only, mode 0o600). This reduces the risk of exposure from other local users or processes. Instead of using Path.write_text, which does not set file permissions, we should use os.open with the appropriate mode, then write the passphrase, and close the file. This change should be made in the main function where the temporary file is created and written. No change is needed for the key file (private.key), as it is not flagged as a password. The GPG command should continue to read from the secure file.

This may require importing os (already present). The code block to replace concerns the writing of the passphrase file.

To fix this, the module-level print should be replaced by a logging call, specifically using the logging module, as recommended. We should add an import for logging (if not already present) at the top of the file, and replace line 23 with logging.error(...). This will avoid writing directly to standard out/consoles during import, and allow users to configure log handling. All other print statements in this file (lines 51, 64, 70, 71, 77, 92, 97, 98) are inside functions—not at module level—so only module-level code needs to be changed. Only change file tools/ci_build/github/windows/jar_packaging.py, specifically lines near 23; add import logging after standard imports if necessary, and replace the print.

[Edward Chen (edgchen1)]

should we also update this react-native-ci.yml job to use an older CMake for the MacOS build?

onnxruntime/tools/ci_build/github/azure-pipelines/templates/react-native-ci.yml

Line 70 in 2132530

- job: ReactNative_CI_iOS_build

[Changming Sun (snnn)]

should we also update this react-native-ci.yml job to use an older CMake for the MacOS build?

onnxruntime/tools/ci_build/github/azure-pipelines/templates/react-native-ci.yml

Line 70 in 2132530

- job: ReactNative_CI_iOS_build

Will address it after this PR.

[Changming Sun (snnn)]

Should be fairly easy to migrate the pipeline to GH by employing an AI agent.