Skip to content

Which properties are required for Microsoft.Testing.Extensions.TrxReport? #12436

Open

Description

xUnit.net has a user who is reporting that the TrxReport extension crashes when a TestNode is reported without TrxFullyQualifiedTypeNameProperty.

Setting aside the specific bug (which is already resolved), it does bring up the idea that xUnit.net does not require test cases to have source information, and in fact supports the idea of test cases that don't come directly from source code at all (and therefore would not have a class or method). I have not put together a full repro yet, but I found it surprising that the lack of this property might cause the TRX report extension to crash, so I wanted to ask:

Which properties are required vs. optional?

Things we report unconditionally:

  • Standard
    • TimingProperty
    • Plus one of:
      • ErrorTestNodeStateProperty (only for errors)
      • FailedTestNodeStateProperty (only for assertion failures)
      • TimeoutTestNodeStateProperty (only for timeout failures)
      • SkippedTestNodeStateProperty (only for skipped/filtered tests)
      • PassedTestNodeStateProperty (only for passed tests)
  • TRX-specific
    • TrxTestDefinitionName

Things which may or may not be reported:

  • Standard
    • TestMethodIdentifierProperty
    • StandardOutputProperty
    • TestFileLocationProperty
    • TestMetadataProperty
  • TRX-specific
    • TrxFullyQualifiedTypeNameProperty
    • TrxCategoriesProperty
    • TrxMessagesProperty
    • TrxExceptionProperty

This is the first time I've seen a crash reported for TRX, so I'm curious whether there are other potentially hidden crashes here. Also, is this documented somewhere that I missed? Should the TRX report extension be more resilient to missing expected properties?

Activity

  1. Evangelink commented on Nov 13, 2026

    @Evangelink
    Member

    Thanks Brad Wilson (@bradwilson). I checked 2.5.1 and this revision of main. The current implementation does have an identity requirement that is problematic for source-less tests, but most of the properties you listed are optional.

    Required by the current implementation

    Besides TestNode.Uid and DisplayName, a result needs one final state: passed, failed, error, timeout, or skipped. Updates without a state, or with discovered/in-progress state, are ignored by TRX.

    When creating a test definition, the reporter also needs either TrxFullyQualifiedTypeNameProperty or TestMethodIdentifierProperty:

    • TrxFullyQualifiedTypeNameProperty supplies the TRX class name.
    • Otherwise, the class name is constructed from the method identifier's namespace and type name.
    • The method identifier supplies the method name when available; otherwise, DisplayName is used.

    The reporter does not check whether that class or method actually exists. A synthetic class label works, and source-file information is not required.

    Optional properties and what they control

    Property TRX behavior
    TimingProperty Its GlobalTiming supplies the test's start time, end time, and duration. Without it, TRX writes zero duration and uses the current UTC time while generating the report for the test's start/end timestamps. These are placeholders, not actual execution times.
    TrxTestDefinitionName Sets the test definition's name. If omitted, DisplayName is used, subject to the repeated-UID consistency check below.
    TrxExceptionProperty Supplies Output/ErrorInfo/Message and StackTrace. Without it, a failed/error/timeout result still has a failed outcome, but no TRX error details.
    TrxMessagesProperty Contains StandardOutputTrxMessage, StandardErrorTrxMessage, and DebugOrTraceTrxMessage, which populate the result's StdOut, StdErr, and DebugTrace respectively.
    TrxCategoriesProperty Adds categories to the test definition through TestCategory/TestCategoryItem.
    TestMetadataProperty The keys Owner, Priority, and Description populate the corresponding test-definition fields. Priority must parse as an integer; the default int.MaxValue is omitted. Other keys become custom key/value entries under Properties/Property.
    TrxWorkItemsProperty Adds associated work-item IDs to the test definition through Workitems/WorkItem.
    FileArtifactProperty Attaches files, such as logs or screenshots, to an individual result. The reporter copies them into the results directory layout and references them through UnitTestResult/ResultFiles/ResultFile. This is unrelated to the test's source-file location.

    StandardOutputProperty and TestFileLocationProperty are not consumed by the TRX reporter. For TRX standard output, the relevant property is TrxMessagesProperty containing StandardOutputTrxMessage.

    Confirmed failures

    I reproduced two report-generation crashes in both versions:

    • When both identity properties are absent, GetClassAndMethodName throws InvalidOperationException. Supplying the identity only during discovery does not help; it is not carried forward to the final update.
    • For repeated results with the same UID, mixing an explicit TrxTestDefinitionName with an omitted one throws if the omitted value's DisplayName fallback differs from the explicit name. Consistently omitting the property does not trigger that check.

    Both cases produced parent exit code 7 and a TRX with no test results. Omitting the other optional properties individually did not crash.

    The ITrxReportCapability documentation is outdated: it lists legacy string keys and implies exception metadata is required. We should document the typed properties, mappings, and fallback rules.

    I agree that the reporter should be more resilient, but avoiding a crash must not silently turn genuinely inconsistent data into a successful report. My preference would be to support source-less tests through a documented fallback, and handle actual contract violations as controlled reporting failures: preserve recoverable results, mark the run unsuccessful, record the affected test IDs and fields in TRX ResultSummary/RunInfos/RunInfo, and return a nonzero exit code without changing the original test outcomes. A console warning or TRX diagnostic alone would not be sufficient for CI consumers that ignore those messages.

  2. added
    area/trxTRX report extension.
    type/questionQuestion or request for clarification.
    and removed
    needs/triageNeeds triage by a maintainer.
    on Nov 13, 2026
  3. bradwilson commented on Nov 13, 2026

    @bradwilson
    Author

    I will update to always supply TrxFullyQualifiedTypeNameProperty, falling back to our unique ID when the type name is not available.

  4. added a commit that references this issue on Nov 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/trxTRX report extension.type/questionQuestion or request for clarification.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions