Feature request: compile build.rs scripts in release mode even when doing a debug build

[TyOverby]

Right now, cargo build compiles build.rs in debug mode, while cargo build --release compiles build.rs in release mode. However build.rs scripts can potentially be very computationally intensive, and I would like the option to run my build.rs script in release mode even when doing a debug compile of the rest of my crate.

As a motivating example, nikomatsakis/lalrpop is a parser generator that does source generation during the custom build step. Just the build.rs step takes 6m30s when compiling in debug, and 33s when compiling in release for a complex grammar.

[alexcrichton]

cc @nikomatsakis

I've certainly wanted this in the past as well. The tricky parts I can think of are:

• We need some sort of "nice syntax" for configuring this. We may not want to use [profile] as this may want to be a per-package instead of a per-application setting.
• It's not just the build script that needs to be optimized here, but also the dependencies. Sometimes the dependencies can be re-used with the original application, so we need to rationalize that as well.

[nikomatsakis]

On Tue, Mar 01, 2016 at 10:46:06AM -0800, Alex Crichton wrote:

cc @nikomatsakis

I've certainly wanted this in the past as well. The tricky parts I can think of are:

• We need some sort of "nice syntax" for configuring this. We may not want to use [profile] as this may want to be a per-package instead of a per-application setting.
• It's not just the build script that needs to be optimized here, but also the dependencies. Sometimes the dependencies can be re-used with the original application, so we need to rationalize that as well.

Yeah, I think the main reason I'd not yet opened an issue requesting
this was not having an answer to these questions. It's also unclear to
me what the defaults ought to be: I'd EXPECT that build.rs typically
wants to be "release by default", but I don't know.

I could also imagine this not being something that one configures, but
rather part of the cargo build command. e.g., perhaps build.rs
defaults always to a release build, but you can run cargo build --preprocessor debug or something when you actually want to debug
your build.rs script.

[sfackler]

The story for lalrpop is kind of interesting here. It's fairly expensive to compile, particularly in release mode (~minutes). For small grammars, total build script time is dominated by compiling lalrpop-snap, in which case you'd probably want a release build, but for larger grammars, the situation inverts and you really need an optimized version of lalrpop.

[TyOverby]

cargo build --preprocessor debug

I'd rather not have this. In the scenario where I have

user application depends on my library which uses lalrpop

I'd like for my library to use a release preprocessor always because I know that the trade off is worth it for my library. I don't want to have to tell people "oh, and if you depend on my library, you should really run cargo build --preprocessor=release.

[nikomatsakis]

@TyOverby

I don't want to have to tell people "oh, and if you depend on my library, you should really run cargo build --preprocessor=release.

My proposal was that release would be the default. You would use debug if you knew you wanted to debug the build.rs script (which would presumably not be the common case). But the point is this is a decision that only the end-user is really in a position to make, not any particular dependency.

[nikomatsakis]

@sfackler

The story for lalrpop is kind of interesting here. It's fairly expensive to compile, particularly in release mode (~minutes).

I'd much rather always have release mode. Sure, the first compile is slow, but after that it doesn't change, so who cares? In general I feel like the first time I compile anything in cargo (or virtually any build system) it is slow, because it has to run off and grab a bunch of deps.

[TyOverby]

@nikomatsakis Ok yeah, I agree with this. Running build scripts in release sounds like a good default.

[alexcrichton]

There's a few concerns I'd have about having a default release for build dependencies:

• If we still have the option to switch to debug mode, we still have the problem of inventing a syntax for such an operation. Arguably the same syntax at that point could be used for "this should be optimized"
• The problem of dependencies is sort of two-fold. A crate can be used both for a build script and for the final artifact generation. In this case do we optimize it or do we not? Unfortunately either choice we make can be the wrong choice depending on how you look at it.

[Nemikolh]

Considering cross-compilation with cargo, isn't already a thing that build dependencies are compiled in a very different way than crate ones?

Having this scenario in mind, the fact that build script are compiled in release mode with the triplet target of the host is a default that make more sense than being similar to the crate compilation profile.

In case a crate is used for both dependencies, for a cross-compilation use case, you would have to compile it twice anyway so it doesn't seems crazy.

[nikomatsakis]

If we still have the option to switch to debug mode, we still have the
problem of inventing a syntax for such an operation. Arguably the same
syntax at that point could be used for "this should be optimized"

I contend that this is the wrong default -- I think the number of
people writing custom logic in their build.rs that they plan to debug
pales in comparison to the numbe of people injecting logic written by
others.

That said, it may be that the right fix here is to stop using
build.rs altogether. That is, maybe what we want is a way to declare a
"preprocess-dependency" that compiles and executes the crate as a
preprocessing step automatically, without the need for a user to write
their own build.rs script. We could then make the default for that
kind of dependency be release, and keep build.rs as it is today.

This also makes using LALRPOP, syntex, and other similar preprocessors
more ergonomic: you only have to add one line to Cargo.toml, instead
of following several distinct steps.

The problem of dependencies is sort of two-fold. A crate can be used both
for a build script and for the final artifact generation. In this case do
we optimize it or do we not? Unfortunately either choice we make can be
the wrong choice depending on how you look at it.

My assumption was that the "build-dependencies" are distinct from the
"main" dependencies. That is, build.rs is a distinct target, so it's
not clear why it has (or should) to use the same build of the crate?

(Feels a lot like workspaces. :)

[alexcrichton]

Yeah this may indeed be a different class of dependencies (preprocessors), but those also sound a lot like plugins...

When you're compiling for the host target (e.g. you didn't pass --target) then if the application depends on a a crate which is also depended on by a build script, we'll only compile that crate once (to avoid unnecessary compilations). I guess we could just decide to compile it twice though

[nikomatsakis]

Yeah this may indeed be a different class of dependencies (preprocessors), but those also sound a lot like plugins...

Is this surprising? I guess you're saying that adding something more "first class" than build.rs doesn't feel right to you, given that we will (eventually) have some kind of plugin system? I could understand that. I think it may well be that we continue to want "preprocessors" of the current form, which generate "all new" source rather than being triggered by some macro use embedded in a .rs file, but I'm not entirely sure about that. It's not clear why that would necessarily be better than some kind of #![invoke_preprocessor] attribute (or maybe even something that triggers automatically when given an extern crate).

[alexcrichton]

Hm yeah I guess they're different enough to warrant different systems. Seems plausible as becoming a thing!