Auto merge of #152677 - ehuss:bootstrap-json-target-spec, r=davidtwco,jieyouxu

Support JSON target specs in bootstrap

JSON target specs were destabilized in #150151 and #151534. However, this broke trying to build rustc itself with a JSON target spec. This is because in a few places bootstrap is manually calling `rustc` without the ability for the user to provide additional flags (primarily, `-Zunstable-options` to enable JSON targets).

There's a few different ways to fix this. One would be to change these calls to `rustc` to include flags provided by the user (such as `RUSTFLAGS_NOT_BOOTSTRAP`). Just to keep things simple, this PR proposes to just unconditionally pass `-Zunstable-options`.

Another consideration here is how maintainable this is. A possible improvement here would be to have a function somewhere (BootstrapCommand, TargetSelection, free function) that would handle appropriately adding the `--target` flag. For example, that's what cargo does in [`CompileKind::add_target_arg`](https://github.com/rust-lang/cargo/blob/592058c7ce08a2ba2628b01e7dc4ce1e72b6bdff/src/cargo/core/compiler/compile_kind.rs#L144-L154).

I have only tested building the compiler and a few tools like rustdoc. I have not tested doing things like building other tools, running tests, etc.

This would be much easier if there was a Docker image for testing the use case of building rustc with a custom target spec (and even better if that ran in CI).

After the next beta branch, using target JSON specs will become more cumbersome because target specs with the `.json` extension will now require passing `-Zjson-target-spec` (from
rust-lang/cargo#16557). This does not affect target specs without the `.json` extension (such as those from RUST_TARGET_PATH). From my testing, it should be sufficient to pass `CARGOFLAGS_NOT_BOOTSTRAP="-Zjson-target-spec"`. I think that should be fine, since this is not a particularly common use case AFAIK. We could extend bootstrap to auto-detect if the target is a file path, and pass `-Zjson-target-spec` appropriately. I tried something similar in ehuss@f0bdd35, which could be adapted if desired.

It would be nice if all of this is documented somewhere. https://rustc-dev-guide.rust-lang.org/building/new-target.html does not really say how to build the compiler with a custom json target.

Fixes #151729

Author: bors

src/bootstrap/src/core/build_steps/compile.rs

@@ -543,6 +543,9 @@ pub fn std_cargo(
         // `MACOSX_DEPLOYMENT_TARGET`, `IPHONEOS_DEPLOYMENT_TARGET`, etc.
         let mut cmd = builder.rustc_cmd(cargo.compiler());
         cmd.arg("--target").arg(target.rustc_target_arg());
+        // FIXME(#152709): -Zunstable-options is to handle JSON targets.
+        // Remove when JSON targets are stabilized.
+        cmd.arg("-Zunstable-options").env("RUSTC_BOOTSTRAP", "1");
         cmd.arg("--print=deployment-target");
         let output = cmd.run_capture_stdout(builder).stdout();
 

src/bootstrap/src/core/builder/cargo.rs

@@ -773,6 +773,10 @@ impl Builder<'_> {
                         .rustc_cmd(compiler)
                         .arg("--target")
                         .arg(target.rustc_target_arg())
+                        // FIXME(#152709): -Zunstable-options is to handle JSON targets.
+                        // Remove when JSON targets are stabilized.
+                        .arg("-Zunstable-options")
+                        .env("RUSTC_BOOTSTRAP", "1")
                         .arg("--print=file-names")
                         .arg("--crate-type=proc-macro")
                         .arg("-")