Skip to content

Skip a single bad executor instead of failing all executor loading - #16239

Merged
Jakub Jareš (nohwnd) merged 1 commit into
microsoft:mainfrom
nohwnd:nohwnd-resilient-executor-invocation
Jul 9, 2026
Merged

Jakub Jareš (nohwnd) merged 1 commit into
microsoft:mainfrom
nohwnd:nohwnd-resilient-executor-invocation

Conversation

@nohwnd

Copy link
Copy Markdown
Member

An ITestExecutor2 that can't be instantiated (for example one without a parameterless constructor) throws while merging the ITestExecutor and ITestExecutor2 lists in MergeTestExtensionLists. That throw tears down the whole executor extension manager: GetExecutorExtensionManager catches it and returns null for the entire assembly, so every executor in that assembly — including the real one — becomes unreachable. The run ends with Could not find test executor 'executor://mstestadapter/v2' and 0 tests run. Discovery survives because it doesn't force instantiation.

The fix skips the one bad executor instead of failing the whole merge, and applies the same per-item resilience to the other loops that eagerly instantiate extensions:

  • MergeTestExtensionLists — the root cause. Wrap .Value per extension and keep the already-cached good version on failure.
  • LoadAndInitializeAllExtensions for executors, discoverers, and settings providers — move the try/catch inside the loop so one bad extension doesn't stop the rest. It still rethrows when the caller asked to fail on error.
  • BaseRunTests debugger-attach loop — a rogue .Value there no longer aborts before the already-resilient execution loop.

The regression test lives in TestExecutorExtensionManagerTests, with a good and a rogue ITestExecutor2 hosted in TestUtilities so it stays isolated from the tests that assert every discovered executor is created. It throws MissingMethodException without the merge fix and passes with it.

A test executor that can't be instantiated (for example one without a
parameterless constructor) used to throw while merging the ITestExecutor
and ITestExecutor2 lists. That tore down the whole executor extension
manager, so GetExecutorExtensionManager returned null for the entire
assembly and every executor in it - including the real one - became
unreachable, ending in "Could not find test executor" and 0 tests run.

Wrap the per-extension instantiation in MergeTestExtensionLists so one
bad executor is skipped rather than failing the whole merge, and apply
the same per-item resilience to the LoadAndInitializeAllExtensions loops
(executor, discovery, settings) and to the debugger-attach probe in
BaseRunTests. Add a regression test with a rogue ITestExecutor2 hosted
in TestUtilities.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings July 8, 2026 16:29

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR hardens vstest’s extension loading so that a single misconfigured/broken extension (e.g., an ITestExecutor2 without a parameterless constructor) doesn’t prevent other valid extensions in the same assembly from being discovered and used. This improves reliability of adapter/executor discovery and execution in the presence of “rogue” extensions.

Changes:

  • Make TestExecutorExtensionManager.MergeTestExtensionLists resilient by handling instantiation failures per extension and continuing the merge.
  • Move try/catch inside “eager instantiation” loops for executors, discoverers, and settings providers so one failing extension doesn’t block the rest (unless the caller opts into failing).
  • Prevent a rogue executor from aborting the debugger-attach pre-pass in BaseRunTests, allowing remaining executors to proceed.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
test/Microsoft.TestPlatform.TestUtilities/ResilientExecutorFixtures.cs Adds a “good” and “rogue” ITestExecutor2 fixture to reproduce instantiation failures in a controlled assembly.
test/Microsoft.TestPlatform.Common.UnitTests/ExtensionFramework/TestExecutorExtensionManagerTests.cs Adds a regression test ensuring executor extension manager creation remains resilient and keeps good executors discoverable.
src/Microsoft.TestPlatform.CrossPlatEngine/Execution/BaseRunTests.cs Wraps debugger-attach condition evaluation to avoid aborting the run when an executor instantiation fails early.
src/Microsoft.TestPlatform.Common/SettingsProvider/SettingsProviderExtensionManager.cs Adjusts Load/Initialize behavior to be per-provider resilient, optionally failing when requested.
src/Microsoft.TestPlatform.Common/ExtensionFramework/TestExecutorExtensionManager.cs Adds per-extension resilience to merge and eager loading paths to avoid whole-manager failure.
src/Microsoft.TestPlatform.Common/ExtensionFramework/TestDiscoveryExtensionManager.cs Makes discoverer initialization resilient per discoverer, while preserving fail-fast when requested.

Comment on lines 146 to 147
// Note: - The below Verbose call should not be under IsVerboseEnabled check as we want to
// call executor.Value even if logging is not enabled.
Comment on lines +4 to +6
using System;
using System.Collections.Generic;

/// <see cref="ITestExecutor2"/> cannot be instantiated.
///
/// <para>
/// These types intentionally live in this non test assembly so that they are only discovered when a
@nohwnd
Jakub Jareš (nohwnd) merged commit a857a5c into microsoft:main Jul 9, 2026
20 checks passed
This was referenced Sep 9, 2026
ptr727-codegen Bot added a commit to ptr727/Utilities that referenced this pull request Sep 9, 2026
Updated
[Microsoft.Extensions.Logging.Abstractions](https://github.com/dotnet/dotnet)
from 10.0.11 to 10.0.12.

<details>
<summary>Release notes</summary>

_Sourced from [Microsoft.Extensions.Logging.Abstractions's
releases](https://github.com/dotnet/dotnet/releases)._

No release notes found for this version range.

Commits viewable in [compare
view](https://github.com/dotnet/dotnet/commits).
</details>

Updated [Microsoft.NET.Test.Sdk](https://github.com/microsoft/vstest)
from 18.9.0 to 18.10.0.

<details>
<summary>Release notes</summary>

_Sourced from [Microsoft.NET.Test.Sdk's
releases](https://github.com/microsoft/vstest/releases)._

## 18.10.0


## What's Changed
* Drop Mono fallback, run .NET Framework tests on Windows only by
@​nohwnd in microsoft/vstest#16158
* Run Microsoft.Testing.Platform test apps under `vstest.console` and
datacollector by @​nohwnd in
microsoft/vstest#16201
* Fix test output eaten by MSBuild terminal logger by @​nohwnd in
microsoft/vstest#16223
* Remove the experimental test session feature by @​nohwnd in
microsoft/vstest#16231
* Skip a single bad executor instead of failing all executor loading by
@​nohwnd in microsoft/vstest#16239
* Assert apartment state instead of using Clipboard in UI tests by
@​nohwnd in microsoft/vstest#16270
* Surface test host crashes during protocol negotiation by @​nohwnd in
microsoft/vstest#16285
* Pass the inferred target platform to the host in run settings by
@​nohwnd in microsoft/vstest#16271
* Report raw invalid IsTargetPlatformInferred value, cover host x64
forcing by @​nohwnd in microsoft/vstest#16295
* Disable the MTP testhost by default (#​16337) by @​nohwnd in
microsoft/vstest#16341


**Full Changelog**:
microsoft/vstest@v18.9.0...v18.10.0

Commits viewable in [compare
view](microsoft/vstest@v18.9.0...v18.10.0).
</details>

Updated [Microsoft.SourceLink.GitHub](https://github.com/dotnet/dotnet)
from 10.0.400 to 10.0.401.

<details>
<summary>Release notes</summary>

_Sourced from [Microsoft.SourceLink.GitHub's
releases](https://github.com/dotnet/dotnet/releases)._

## 10.0.401

[Release](https://github.com/dotnet/core/releases/tag/v10.0.12)

## What's Changed
* [release/10.0.4xx] Source code updates from dotnet/sdk by
@​dotnet-maestro[bot] in dotnet/dotnet#8011
* [release/10.0.4xx] Source code updates from dotnet/sdk by
@​dotnet-maestro[bot] in dotnet/dotnet#8037
* [release/10.0.4xx] Source code updates from dotnet/sdk by
@​dotnet-maestro[bot] in dotnet/dotnet#8038
* Update Source-Build SDK Diff Tests Baselines and Exclusions by
@​dotnet-sb-bot in dotnet/dotnet#7754
* [release/10.0.4xx] Source code updates from dotnet/arcade by
@​dotnet-maestro[bot] in dotnet/dotnet#8101
* [release/10.0.4xx] Source code updates from dotnet/source-build-assets
by @​dotnet-maestro[bot] in dotnet/dotnet#8104
* [release/10.0.4xx] Source code updates from dotnet/msbuild by
@​dotnet-maestro[bot] in dotnet/dotnet#8142
* [release/10.0.4xx] Source code updates from dotnet/sdk by
@​dotnet-maestro[bot] in dotnet/dotnet#8132
* [release/10.0.4xx] Source code updates from dotnet/source-build-assets
by @​dotnet-maestro[bot] in dotnet/dotnet#8127
* [release/10.0.4xx] Source code updates from dotnet/arcade by
@​dotnet-maestro[bot] in dotnet/dotnet#8156
* [release/10.0.4xx] Source code updates from dotnet/roslyn by
@​dotnet-maestro[bot] in dotnet/dotnet#8148
* [release/10.0.4xx] Source code updates from dotnet/sdk by
@​dotnet-maestro[bot] in dotnet/dotnet#8176
* [release/10.0.4xx] Source code updates from dotnet/arcade by
@​dotnet-maestro[bot] in dotnet/dotnet#8171
* [release/10.0.4xx] Source code updates from dotnet/scenario-tests by
@​dotnet-maestro[bot] in dotnet/dotnet#8092
* [release/10.0.4xx] Fix IdentityModel shared-components source build by
@​mthalman in dotnet/dotnet#8189
* [release/10.0.4xx] Source code updates from nuget/nuget.client by
@​dotnet-maestro[bot] in dotnet/dotnet#8201
* [release/10.0.4xx] Allow submodule changes when reseting submodules by
@​github-actions[bot] in dotnet/dotnet#8223
* [release/10.0.4xx] Source code updates from dotnet/roslyn by
@​dotnet-maestro[bot] in dotnet/dotnet#8215
* [release/10.0.4xx] Source code updates from dotnet/arcade by
@​dotnet-maestro[bot] in dotnet/dotnet#8210
* [release/10.0.4xx] Cloak SignatureCycle2 binary from VMR by @​mthalman
in dotnet/dotnet#8226
* [release/10.0.4xx] Ignore spurious SBoM warnings by @​akoeplinger in
dotnet/dotnet#8165
* [release/10.0.4xx] Source code updates from dotnet/arcade by
@​dotnet-maestro[bot] in dotnet/dotnet#8231
* Update to the 10.0.11 August runtimes from build 324376 by @​mthalman
in dotnet/dotnet#8259
* Merging internal commits for release/10.0.4xx by @​dotnet-bot in
dotnet/dotnet#8242
* [release/10.0.4xx] Source code updates from dotnet/arcade by
@​dotnet-maestro[bot] in dotnet/dotnet#8278
* [release/10.0.4xx] Source code updates from dotnet/arcade by
@​dotnet-maestro[bot] in dotnet/dotnet#8304
* [release/10.0.4xx] Source code updates from dotnet/roslyn by
@​dotnet-maestro[bot] in dotnet/dotnet#8347
* [release/10.0.4xx] Remove duplicate dependency from scenario-tests by
@​dkurepa in dotnet/dotnet#8350
* [release/10.0.4xx] Source code updates from dotnet/sourcelink by
@​dotnet-maestro[bot] in dotnet/dotnet#8344
* [release/10.0.4xx] Source code updates from dotnet/templating by
@​dotnet-maestro[bot] in dotnet/dotnet#8354
* [release/10.0.4xx] Fix SignCheckExclusionsFile.txt to allow signed
hotreload .js file by @​github-actions[bot] in
dotnet/dotnet#8383
* [release/10.0.4xx] Remove TSA bug filing from pipelines by
@​akoeplinger in dotnet/dotnet#8389
* [release/10.0.4xx] Source code updates from dotnet/arcade by
@​dotnet-maestro[bot] in dotnet/dotnet#8396
* [release/10.0.4xx] [release/10.0.1xx] Update binskim lastModifiedDate
in PipelineAutobaseliningConfig.yml by @​github-actions[bot] in
dotnet/dotnet#8408
* [release/10.0.4xx] Fix SignCheckExclusionsFile.txt for .js in
source-build artifacts by @​github-actions[bot] in
dotnet/dotnet#8405
* [release/10.0.4xx] Source code updates from dotnet/templating by
@​dotnet-maestro[bot] in dotnet/dotnet#8419
* [release/10.0.4xx] Source code updates from dotnet/templating by
@​dotnet-maestro[bot] in dotnet/dotnet#8422
* [release/10.0.4xx] Source code updates from dotnet/msbuild by
@​dotnet-maestro[bot] in dotnet/dotnet#8429
* [release/10.0.4xx] Source code updates from dotnet/source-build-assets
by @​dotnet-maestro[bot] in dotnet/dotnet#8313
* [release/10.0.4xx] Source code updates from dotnet/templating by
@​dotnet-maestro[bot] in dotnet/dotnet#8452
* [release/10.0.4xx] Source code updates from dotnet/source-build-assets
by @​dotnet-maestro[bot] in dotnet/dotnet#8453
* [release/10.0.4xx] Source code updates from dotnet/msbuild by
@​dotnet-maestro[bot] in dotnet/dotnet#8469
* [release/10.0.4xx] Source code updates from dotnet/fsharp by
@​dotnet-maestro[bot] in dotnet/dotnet#8521
* [release/10.0.4xx] Source code updates from dotnet/arcade by
@​dotnet-maestro[bot] in dotnet/dotnet#8574
* [release/10.0.4xx] Bump patch version to 10.0.401 by @​vseanreesermsft
in dotnet/dotnet#8577


 ... (truncated)

Commits viewable in [compare
view](dotnet/dotnet@v10.0.400...v10.0.401).
</details>

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore <dependency name> major version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's major version (unless you unignore this specific
dependency's major version or upgrade to it yourself)
- `@dependabot ignore <dependency name> minor version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's minor version (unless you unignore this specific
dependency's minor version or upgrade to it yourself)
- `@dependabot ignore <dependency name>` will close this group update PR
and stop Dependabot creating any more for the specific dependency
(unless you unignore this specific dependency or upgrade to it yourself)
- `@dependabot unignore <dependency name>` will remove all of the ignore
conditions of the specified dependency
- `@dependabot unignore <dependency name> <ignore condition>` will
remove the ignore condition of the specified dependency and ignore
conditions


</details>
This was referenced Nov 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants