Fixture support

[epage]

The idea is to print pytest-like fixtures to Rust, like test-r and rustest do.

Fixtures should allow composable test setup and tear down

• Fixtures can take fixtures
• Fixtures can be paramaterized (Parametrized tests support #19)
• Fixtures can be ignored, causing dependent tests to be Skipped (#[ignore] macro support #16)
• Fixtures can panic, causing dependent tests to be Skipped (#[should_panic] macro support #15)
• Teardown is down through the type returned
  • Like rustest, we offer a teardown attribute. Should probably be named drop
  • Once we have generators in Rust, we could consider using those for setup/teardown phases
• Fixtures can be scoped to the parameter, test, or global

Differences from Pytest

• Like rustest, fixtures should be found by their type through the type system, rather than behind the scenes machinery like pytest.
• While fixtures can be parameterized (Parametrized tests support #19), fixtures can't enumerate parameters. Pytest allows parameterization through both markers and fixtures while we'll keep these separate. If nothing else, this will give a clear demarcation point for what runs during Discovery Phase (parameter discovery) vs Run Phase (fixtures)

[skogseth]

I've played around a little bit with the code and gotten familiar, is this an issue I can work on? And if so, are there any designs laid out for this? Is the idea to have a #[fixture] attribute?

[epage]

Thanks for wanting to help!

I filled out the issue with more ideas of where I was thinking of going. This can all be developed incrementally, a PR for each piece.

My initial thought was to focus on feature parity (#13, #15, #16) and then move on to what Cargo would need for testing (#48, maybe pre-run hook). My thought was to then work on #19 first to see how that influences fixtures.

However, if you want to dig into fixtures first, feel free. Some quick thoughts on the nitty gritty

• I'm trying to avoid proc-macro machiner for when attribute declarative macros are stabilized and to keep the proc-macro deps light weight
• We need to work out how fixtures get reported in the JSON output. Early on this will show up in terms of fixture failure reporting. Do we fail on the tests? Do we enumerate fixtures independent of tests and Skip the tests? What about when we add #[should_panic] so that the fixture didn't return anything, how do we report that?
• In general, my aim is for macros to be wrappers around a regular API so we have something concrete we can point people to, like claps derive API and clap_builder.
  • We can explore Deref Specialization, Tacit Traits, and magic handler functions for implementing a lot of this outside of macros.

[skogseth]

Okay, thanks for filling out the issue, and giving me a rundown of the current plan. I did notice that the majority of the macros "work" is in the declarative ones, at first glance a bit surprising, but it's been very pleasant to work with when experimenting. I'll use the same strategy with whatever I need to add that's macro related 👍 I'll try to take a look at either #15 or #16 then.