You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Which properties are required for Microsoft.Testing.Extensions.TrxReport? #12436
Setting aside the specific bug (which is already resolved), it does bring up the idea that xUnit.net does not require test cases to have source information, and in fact supports the idea of test cases that don't come directly from source code at all (and therefore would not have a class or method). I have not put together a full repro yet, but I found it surprising that the lack of this property might cause the TRX report extension to crash, so I wanted to ask:
Which properties are required vs. optional?
Things we report unconditionally:
Standard
TimingProperty
Plus one of:
ErrorTestNodeStateProperty (only for errors)
FailedTestNodeStateProperty (only for assertion failures)
TimeoutTestNodeStateProperty (only for timeout failures)
SkippedTestNodeStateProperty (only for skipped/filtered tests)
PassedTestNodeStateProperty (only for passed tests)
TRX-specific
TrxTestDefinitionName
Things which may or may not be reported:
Standard
TestMethodIdentifierProperty
StandardOutputProperty
TestFileLocationProperty
TestMetadataProperty
TRX-specific
TrxFullyQualifiedTypeNameProperty
TrxCategoriesProperty
TrxMessagesProperty
TrxExceptionProperty
This is the first time I've seen a crash reported for TRX, so I'm curious whether there are other potentially hidden crashes here. Also, is this documented somewhere that I missed? Should the TRX report extension be more resilient to missing expected properties?
Thanks Brad Wilson (@bradwilson). I checked 2.5.1 and this revision of main. The current implementation does have an identity requirement that is problematic for source-less tests, but most of the properties you listed are optional.
Required by the current implementation
Besides TestNode.Uid and DisplayName, a result needs one final state: passed, failed, error, timeout, or skipped. Updates without a state, or with discovered/in-progress state, are ignored by TRX.
When creating a test definition, the reporter also needs either TrxFullyQualifiedTypeNameProperty or TestMethodIdentifierProperty:
TrxFullyQualifiedTypeNameProperty supplies the TRX class name.
Otherwise, the class name is constructed from the method identifier's namespace and type name.
The method identifier supplies the method name when available; otherwise, DisplayName is used.
The reporter does not check whether that class or method actually exists. A synthetic class label works, and source-file information is not required.
Optional properties and what they control
Property
TRX behavior
TimingProperty
Its GlobalTiming supplies the test's start time, end time, and duration. Without it, TRX writes zero duration and uses the current UTC time while generating the report for the test's start/end timestamps. These are placeholders, not actual execution times.
TrxTestDefinitionName
Sets the test definition's name. If omitted, DisplayName is used, subject to the repeated-UID consistency check below.
TrxExceptionProperty
Supplies Output/ErrorInfo/Message and StackTrace. Without it, a failed/error/timeout result still has a failed outcome, but no TRX error details.
TrxMessagesProperty
Contains StandardOutputTrxMessage, StandardErrorTrxMessage, and DebugOrTraceTrxMessage, which populate the result's StdOut, StdErr, and DebugTrace respectively.
TrxCategoriesProperty
Adds categories to the test definition through TestCategory/TestCategoryItem.
TestMetadataProperty
The keys Owner, Priority, and Description populate the corresponding test-definition fields. Priority must parse as an integer; the default int.MaxValue is omitted. Other keys become custom key/value entries under Properties/Property.
TrxWorkItemsProperty
Adds associated work-item IDs to the test definition through Workitems/WorkItem.
FileArtifactProperty
Attaches files, such as logs or screenshots, to an individual result. The reporter copies them into the results directory layout and references them through UnitTestResult/ResultFiles/ResultFile. This is unrelated to the test's source-file location.
StandardOutputProperty and TestFileLocationProperty are not consumed by the TRX reporter. For TRX standard output, the relevant property is TrxMessagesProperty containing StandardOutputTrxMessage.
Confirmed failures
I reproduced two report-generation crashes in both versions:
When both identity properties are absent, GetClassAndMethodName throws InvalidOperationException. Supplying the identity only during discovery does not help; it is not carried forward to the final update.
For repeated results with the same UID, mixing an explicit TrxTestDefinitionName with an omitted one throws if the omitted value's DisplayName fallback differs from the explicit name. Consistently omitting the property does not trigger that check.
Both cases produced parent exit code 7 and a TRX with no test results. Omitting the other optional properties individually did not crash.
The ITrxReportCapability documentation is outdated: it lists legacy string keys and implies exception metadata is required. We should document the typed properties, mappings, and fallback rules.
I agree that the reporter should be more resilient, but avoiding a crash must not silently turn genuinely inconsistent data into a successful report. My preference would be to support source-less tests through a documented fallback, and handle actual contract violations as controlled reporting failures: preserve recoverable results, mark the run unsuccessful, record the affected test IDs and fields in TRX ResultSummary/RunInfos/RunInfo, and return a nonzero exit code without changing the original test outcomes. A console warning or TRX diagnostic alone would not be sufficient for CI consumers that ignore those messages.
Activity
Evangelink commentedon Nov 13, 2026
Thanks Brad Wilson (@bradwilson). I checked 2.5.1 and this revision of main. The current implementation does have an identity requirement that is problematic for source-less tests, but most of the properties you listed are optional.
Required by the current implementation
Besides
TestNode.UidandDisplayName, a result needs one final state: passed, failed, error, timeout, or skipped. Updates without a state, or with discovered/in-progress state, are ignored by TRX.When creating a test definition, the reporter also needs either
TrxFullyQualifiedTypeNamePropertyorTestMethodIdentifierProperty:TrxFullyQualifiedTypeNamePropertysupplies the TRX class name.DisplayNameis used.The reporter does not check whether that class or method actually exists. A synthetic class label works, and source-file information is not required.
Optional properties and what they control
TimingPropertyGlobalTimingsupplies the test's start time, end time, and duration. Without it, TRX writes zero duration and uses the current UTC time while generating the report for the test's start/end timestamps. These are placeholders, not actual execution times.TrxTestDefinitionNameDisplayNameis used, subject to the repeated-UID consistency check below.TrxExceptionPropertyOutput/ErrorInfo/MessageandStackTrace. Without it, a failed/error/timeout result still has a failed outcome, but no TRX error details.TrxMessagesPropertyStandardOutputTrxMessage,StandardErrorTrxMessage, andDebugOrTraceTrxMessage, which populate the result'sStdOut,StdErr, andDebugTracerespectively.TrxCategoriesPropertyTestCategory/TestCategoryItem.TestMetadataPropertyOwner,Priority, andDescriptionpopulate the corresponding test-definition fields. Priority must parse as an integer; the defaultint.MaxValueis omitted. Other keys become custom key/value entries underProperties/Property.TrxWorkItemsPropertyWorkitems/WorkItem.FileArtifactPropertyUnitTestResult/ResultFiles/ResultFile. This is unrelated to the test's source-file location.StandardOutputPropertyandTestFileLocationPropertyare not consumed by the TRX reporter. For TRX standard output, the relevant property isTrxMessagesPropertycontainingStandardOutputTrxMessage.Confirmed failures
I reproduced two report-generation crashes in both versions:
GetClassAndMethodNamethrowsInvalidOperationException. Supplying the identity only during discovery does not help; it is not carried forward to the final update.TrxTestDefinitionNamewith an omitted one throws if the omitted value'sDisplayNamefallback differs from the explicit name. Consistently omitting the property does not trigger that check.Both cases produced parent exit code 7 and a TRX with no test results. Omitting the other optional properties individually did not crash.
The
ITrxReportCapabilitydocumentation is outdated: it lists legacy string keys and implies exception metadata is required. We should document the typed properties, mappings, and fallback rules.I agree that the reporter should be more resilient, but avoiding a crash must not silently turn genuinely inconsistent data into a successful report. My preference would be to support source-less tests through a documented fallback, and handle actual contract violations as controlled reporting failures: preserve recoverable results, mark the run unsuccessful, record the affected test IDs and fields in TRX
ResultSummary/RunInfos/RunInfo, and return a nonzero exit code without changing the original test outcomes. A console warning or TRX diagnostic alone would not be sufficient for CI consumers that ignore those messages.bradwilson commentedon Nov 13, 2026
I will update to always supply
TrxFullyQualifiedTypeNameProperty, falling back to our unique ID when the type name is not available.microsoft/testfx#12436: Add TrxFullyQualifiedTypeNameProperty uncondi…
Use a fallback class name in TRX reports for nodes without type info