Override compile-time test environment paths for relocated test execution

[elyouG]

What does this PR try to resolve?

Cargo already synthesizes several compile-time environment variables for test and bench units in artifact.rs::get_env(), including OUT_DIR, CARGO_MANIFEST_DIR, and CARGO_BIN_EXE_*.

This PR adds optional environment-based overrides for those values when compiling test and bench targets:

• CARGO_TEST_OUT_DIR_OVERRIDE → overrides OUT_DIR
• CARGO_TEST_MANIFEST_DIR_OVERRIDE → overrides CARGO_MANIFEST_DIR
• CARGO_TEST_BIN_EXE_DIR_OVERRIDE → overrides the directory used to construct CARGO_BIN_EXE_

The motivation comes from a downstream packaged-test use case, in particular Rust tests executed through Yocto ptest.

In that setup, test binaries are often:

• Compiled in one environment and filesystem layout
• Installed later into a packaged test directory
• Executed in a different runtime layout than Cargo’s original build tree

In a regular Cargo workflow, compile-time paths such as OUT_DIR, CARGO_MANIFEST_DIR, and CARGO_BIN_EXE_* usually remain valid because tests execute within Cargo’s own build tree. In packaged test environments, this assumption does not always hold:

• The original build-time OUT_DIR may no longer exist
• The original CARGO_MANIFEST_DIR may no longer be the right base path for fixtures or test resources
• The binary under test may be installed elsewhere, for example in the final bindir, while integration tests still rely on env!("CARGO_BIN_EXE_")

As a result, tests that embed these values at compile time can fail even though the tested code itself is correct.

This is not just theoretical. In downstream work, I ran into repeated failures of packaged Rust tests for exactly these reasons. In at least one case (rpm-sequoia), the practical workaround was to add project-specific logic to compensate for invalid runtime paths. The purpose of this PR is to avoid having to solve that class of problem crate-by-crate, and instead provide a small opt-in mechanism directly in Cargo, at the point where Cargo already synthesizes these values.
see rpm-software-management/rpm-sequoia#86

artifact.rs::get_env() is already the place where Cargo constructs these compile-time variables for test and bench units, so it is the natural place to add optional overrides for environments that need to relocate tests outside Cargo’s original build tree.

This makes those paths relocatable when the execution context changes, while preserving Cargo’s existing behavior in the normal case.

The new behavior is strictly opt-in:

• If none of the new _CARGO_TEST* variables are set, Cargo behaves exactly as before
• The overrides apply only to test and bench targets
• Normal library and binary builds are unchanged
• For CARGO_BIN_EXE_*, the existing cargo check fallback behavior is preserved

From a downstream perspective, this helps avoid project-by-project patches whose only purpose is to work around relocated test paths.

For example, instead of modifying a crate’s tests to introduce custom runtime environment variables or alternate path lookup logic, a downstream Yocto recipe can simply set the desired compile-time values directly:

do_compile_ptest_cargo:prepend() {
    os.environ["__CARGO_TEST_OUT_DIR_OVERRIDE"] = d.getVar("libdir")
    os.environ["__CARGO_TEST_MANIFEST_DIR_OVERRIDE"] = d.getVar("PTEST_PATH")
}

do_install_ptest:append () {
    install -d ${D}${PTEST_PATH}/src
    install -m 644 ${S}/src/symbols.txt ${D}${PTEST_PATH}/src/symbols.txt
}

With this kind of setup, the crate can continue using Cargo’s standard variables, and the downstream environment can adapt them when the test execution layout differs from the build layout.

More generally, I think this also makes Cargo’s standard test environment variables more attractive to projects that do not use them yet. If downstream environments can override them when needed, relying on env!("OUT_DIR"), env!("CARGO_MANIFEST_DIR"), and env!("CARGO_BIN_EXE_") becomes easier to support than ad hoc path reconstruction or custom per-project environment variables.

This PR is currently split into 3 commits for readability:

1.override OUT_DIR
2.override CARGO_MANIFEST_DIR
3.override CARGO_BIN_EXE_*

I am happy to squash them if preferred.

How to test and review this PR?

This PR is easiest to review commit-by-commit, since each commit applies the same opt-in pattern to one compile-time test variable.

For each commit, the main review points are:

• The override is only applied to test and bench units
• The override is only used when the corresponding _CARGO_TEST* variable is present
• Cargo’s default behavior is unchanged otherwise

I tested this in two ways.

First, I verified that Cargo’s normal behavior remains unchanged when no override variables are set:

• Regular test builds continue to behave as before
• Normal library and binary builds are unaffected
• Cargo check keeps the existing CARGO_BIN_EXE_* fallback behavior

Second, I verified that the overrides are picked up correctly when the new variables are provided:

• OUT_DIR can be redirected to an alternate path
• CARGO_MANIFEST_DIR can be redirected to an alternate path
• CARGO_BIN_EXE_ can be redirected by overriding the directory used to construct it

I also validated the approach against the downstream use case that motivated this PR: packaged Rust tests in a Yocto ptest workflow, where tests are compiled in one layout and executed later from another installed location. The goal there is to let downstream environments adapt Cargo’s standard compile-time test variables directly, instead of having to ask each affected crate to introduce project-specific workarounds.

Cc: @ycongal-smile

[rustbot]

r? @epage

rustbot has assigned @epage.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Why was this reviewer chosen?

The reviewer was selected based on:

• Owners of files modified in this PR: @ehuss, @epage, @weihanglo
• @ehuss, @epage, @weihanglo expanded to ehuss, epage, weihanglo
• Random selection from ehuss, epage, weihanglo

[epage]

Note that our contrib guide says that PRs should first start as issues that get Accepted by the team before posting PRs. I have a personal write up going into more detail on this at https://epage.github.io/dev/pr-style/#d-issue.

[elyouG]

Thanks for the pointer. I've converted this PR back to draft and opened issue #16817 first to discuss the problem and the proposed direction before requesting review again.