Skip to content

Preserve nested exception diagnostics in MTP JSON, HTML and JUnit reports #12274

Closed

Description

Describe the bug

The shared TestResultCaptureHelper.ExtractExceptionDetails captures a reported exception's outer Message and StackTrace, but not its inner exception chain. For frameworks that publish an exception without a pre-flattened Explanation, CTRF/HTML/JUnit report consumers can lose the root cause that the terminal preserves. A new report format would not fix this capture defect.

Version used

Confirmed in source at b3f3c980c480c5d97976d658ac367b1e246502be in src/Platform/SharedExtensionHelpers/TestResultCaptureHelper.cs. The earlier packed-host observation used development MTP/CTRF packages; this issue does not claim a new reproduction against every released framework or package version.

Steps To Reproduce

  1. Run a registered MTP test framework that publishes a failed node with no separate Explanation and an exception shaped as follows:
new InvalidOperationException(
    "[INVOCATION_FAILED] Invocation failed.",
    new InvalidOperationException("[SERVICE_AUTH_401] The service rejected the credential."))
  1. Enable the existing CTRF, HTML or JUnit report extension.
  2. Compare the failed result's message/diagnostic trace with the full terminal exception details.

Expected behavior

The report preserves the nested cause and all aggregate-exception branches in diagnostic fields, subject to the existing explicit size limits. An explicit framework explanation and the outer exception identity remain intact; ordinary exceptions do not acquire unnecessary formatting changes.

Actual behavior

The report capture falls back to the outer message and copies only the outer stack trace. The inner SERVICE_AUTH_401 diagnostic is absent in the demonstrated producer shape. Selective queries cannot recover information the report never retained.

Additional context

Follow-up to #8824; evidence and its limits are recorded in #8828 (comment). #8828 is closed as not planned; this is a focused correction to existing reporters, not a new JSON package.

Acceptance: focused nested/aggregate/explicit-explanation regressions; unchanged ordinary-exception behavior and existing truncation; package-consuming coverage for each affected report's associated result; the same regression assertions fail for the intended diagnostic omission without the production fix and pass with it.

Activity

  1. added
    area/mtpMicrosoft.Testing.Platform core library.
    area/mtp-reportingShared MTP reporting infrastructure and report formats without a dedicated area label.
    on Nov 9, 2026
  2. Evangelink commented on Nov 9, 2026

    @Evangelink
    MemberAuthor

    Implemented locally in commit d8ae1ef788e7a72fc767ccfca29e58b186ddfa4c on the session branch. This commit has not been pushed and no PR has been opened; the issue remains open pending delivery/merge.

    The shared helper preserves the framework explanation, outer exception type, and ordinary Exception.StackTrace behavior. For an exception with inner causes, its diagnostic trace now uses the full exception representation so nested and aggregate branches survive. Existing message/trace size limits remain in force; no new schema, package or API was introduced. CTRF, HTML and JUnit package documentation is updated.

    Test-value evidence

    Contract Original production source with the same tests Correct implementation
    Shared capture: nested causes across failed/error/timeout/cancelled states, aggregate branches, explicit explanations and trace limits 29 executed: 22 passed, 7 failed at assertions for the omitted diagnostic trace; no skips; exit 2. 29 passed, 0 failed/skipped on both net10.0 and net472.
    Actual packed CTRF/HTML/JUnit consumer Reports_NestedExceptions_PreserveAssociatedCausesAndExplanations executed and failed at the assertion that CTRF result nested must retain its trace; exit 2. Same final test passed; five distinct results were checked by identity, name, outcome and diagnostic association in each report; outer exit 0, intentionally failing child exit 2.

    The unit coverage includes TryCaptureCore_NestedException_PreservesInnerDiagnosticTrace, TryCaptureCore_AggregateException_PreservesAllInnerDiagnostics, TryCaptureCore_ExplanationWithNestedException_PreservesExplanationAndTrace, TryCaptureCore_SimpleThrownException_PreservesOriginalStackTrace, and TryCapture_NestedException_KeepsDiagnosticTraceBounded.

    Each state was independently rebuilt and packed with .\build.cmd -c Release -pack -bl (passed, exit 0; invoked as the Windows repository script). The comparison retained the exact final test bytes on original source b3f3c980c480c5d97976d658ac367b1e246502be in a separate task-owned checkout. The generated net11.0 consumer resolved MTP/CTRF/HTML 3.0.0-dev and JUnit 1.0.0-dev; its staged runtime DLL hashes matched the corresponding package payloads in each state. SDK: repository-pinned 12.0.100-alpha.1.26509.101; outer acceptance host: net12.0.

    Commands used the FullyQualifiedName~TestResultCaptureHelperTests unit selector and FullyQualifiedName~ReportExceptionDiagnosticsTests.Reports_NestedExceptions_PreserveAssociatedCausesAndExplanations acceptance selector, with isolated TRX/results directories. The final acceptance selection was independently discovered and its executed/passed count was verified from fresh TRX.

    The actual generated HTML report was opened and its nested failure expanded: the escaped inner message rendered as literal text and remained associated with NestedFailure. No layout code changed. Local logs, paired reports, hashes and binlogs are retained; they are not a published benchmark artifact. CI, other operating systems and release-branch/backport validation are not being claimed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/mtpMicrosoft.Testing.Platform core library.area/mtp-reportingShared MTP reporting infrastructure and report formats without a dedicated area label.

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions