Put testset exceptions in the test record.

[maleadt]

This makes it easier to differentiate between failures and crashes.

Should also make it easier to switch to Malt.jl which doesn't automatically serialize thrown exceptions (which would be bad in order to reconstruct testset failures).

[vchuravy]

One thing we should check is if throwing a custom exception works.

Something like:

struct MyError <: Exception end
throw(MyError())

I assume (partially based on the Malt docs), that exceptions that are only defined on the worker process would lead to a serialization failure, which we should document as a caveat. Thankfully the 1.0 interface requires the module and thus encourages the user to load the package on the main process, so it should only occur for exceptions for package that are only loaded on the worker, or newly defined on the worker.

[maleadt]

I'm only storing TestSetExceptions though, all others are rethrown (and would get converted to Strings by Malt).

[christiangnrd]

Could this still print the time? It's sometimes helpful to know how long it takes to crash

[maleadt]

Crashes are highly unlikely, since we now catch every exception worker-side, so I rather wanted to keep the reporting logic as simple as possible. Besides, the testset right below will always report the time it took, even without --verbose:

               │          │ ──────────────── CPU ──────────────── │
Test  (Worker) │ Time (s) │ GC (s) │ GC % │ Alloc (MB) │ RSS (MB) │
basic      (2) |         crashed at 2025-10-15T08:32:27.239

Test Summary: | Error  Total  Time
  Overall     |     1      1  5.6s
    basic     |     1      1  3.8s
    FAILURE

[maleadt]

One thing we should check is if throwing a custom exception works.

Something like:

struct MyError <: Exception end
throw(MyError())

Interestingly, this now works as-is, even when the main process doesn't have this definition. Try adding that to test/basic.jl, which the test runner never includes (only the worker), yet we still get to see:

Error in testset basic:
Error During Test at /Users/tim/Julia/pkg/ParallelTestRunner/src/ParallelTestRunner.jl:233
  Got exception outside of a @test
  LoadError: Main.var"##basic#232".MyError()

I'm not sure how that works? Is it actually serializing the struct definition too? This is probably something Malt doesn't support.