Add a "cargo bench --debug" option

[asomers]

A mere cargo test neither compiles nor runs the benchmarks. In order to keep them from bitrotting, they must be built and run in CI. But release builds take much more time and RAM than debug builds. In a CI situation, the benchmark results aren't important; only the fact that they can run without errors. So it would be very helpful if there were a cargo bench --debug option

The closest that can be done right now is to do cargo build --benches && target/debug/<my_benchmark>-*. That usage is problematic because

1. The CI script must be updated everytime a new benchmark program is added, and
2. That invocation will run old, stale binaries, too.

Instead, it would be great if cargo had a single cargo bench --debug command that would build the benchmarks in debug mode and then run them.

[droundy]

Any chance there is a workaround on this yet? I've got a bug that shows up only in my benchmark, and it'd be far easier to find with debug symbols enabled.

[sfackler]

You can turn on debug symbols when building benchmarks.

[asomers]

Try cargo +nightly build --benches.

[ehuss]

You can now run cargo test --bench '*' to build and run all of the benchmarks in the benches/ directory, and the libtest test runner will run one iteration to verify that it works.

[asomers]

@ehuss that's a perfect solution!