Confusing CI error when compiler build fails

[RalfJung]

Opening this failed log shows a somewhat confusing error message:

It very much looks like this toolstate error is what makes the build fail. That's wrong though, there is another error hiding inside "Building compiler artifacts", and that is the true error.

Is there a way we can avoid showing errors that are entirely unrelated to the actual problem? Also, is there a way we can avoid hiding the compiler error? AFAIK github doesn't fold a group if the endgroup is missing, so ideally we'd not emit endgroup if there was an error that aborts the build.

Cc @rust-lang/bootstrap

[tgross35]

I feel like this may be a recent change, I have noticed RLA capturing this output a lot since about last week.

[onur-ozkan]

This could be related with one of the BootstrapCommand PRs. cc @Kobzol

[RalfJung]

What's RLA?

[tgross35]

Rust log analyzer. Or Rust length array if we come up with safer VLAs :)

[Kobzol]

This behavior is specific to the checktools.sh script. It essentially ignores errors (set +e), then it tries to build some tools, and then records the output of the build in the toolstate.

So in this case, the actual error that fails the CI job is really

ERROR: Tool `book` was not recorded in tool state.
ERROR: Tool `nomicon` was not recorded in tool state.
ERROR: Tool `reference` was not recorded in tool state.
ERROR: Tool `rust-by-example` was not recorded in tool state.
ERROR: Tool `edition-guide` was not recorded in tool state.
ERROR: Tool `embedded-book` was not recorded in tool state.

not the previous compiler ICE (that is explicitly allowed by the script).

[Kobzol]

To clarify: bootstrap is doing everything correctly here. The compiler build fails, it produces an error, and doesn't end its group. But because checktools.sh uses set +e, it swallows this error, and continues with another bootstrap execution, which opens a new group, which causes GHA to automatically close the previous group containing the error.

[RalfJung]

Ah! That makes sense. Hm, maybe we could do the rustc build before the set +e, to ensure errors there actually count? Ideally we'd convert these into subtrees and then get rid of the entire toolstate system (and the submodule pain)... Though I seem to recall @ehuss actually prefers submodules over subtrees.