Skip to content

Smoke tests failing in stage 2 with VSTESTTASK2 error #4003

Closed

Description

@mthalman
VSTESTTASK2 : error Message:  [/vmr/test/Microsoft.DotNet.SourceBuild.SmokeTests/Microsoft.DotNet.SourceBuild.SmokeTests.csproj]

No other information exists. It's an error contained in the build output but not an actual test case failure. Example build (internal link)

It would seem this is introduced by new functionality from within vstest that gets compiled into the SDK and used in the stage 2 build.

Activity

  1. mthalman commented on Jan 22, 2024

    @mthalman
    MemberAuthor

    This may be a regression caused by microsoft/vstest#4834. These tests were working last Friday and failing today after incorporating changes that included microsoft/vstest#4834. @nohwnd - Can you take a look?
    Here's a binlog: BuildTests_0122193259.zip. We need to get this resolved soon for .NET 9 Preview 1.

  2. added
    ops-monitorIssues created/handled by the source build monitor role
    on Jan 22, 2024
  3. mthalman commented on Jan 23, 2024

    @mthalman
    MemberAuthor

    I ran a test build (internal link) that reverted the change from microsoft/vstest#4834 but that didn't help at all.

  4. nohwnd commented on Jan 23, 2024

    @nohwnd
    Member

    This is more likely linked to this change: https://github.com/dotnet/sdk/pull/38098/files which changes how failed tests are reported to take advantage of msbuild terminal logger which is now the default experience for dotnet build.

    I am trying to repro to see if the smoke tests failed and VSTEST fails to report the details of the failure. Or if VSTEST is failing and the change needs to be reverted.

  5. nohwnd commented on Jan 23, 2024

    @nohwnd
    Member

    Running another test build with opt-out from the new feature https://dev.azure.com/dnceng/internal/_build/results?buildId=2360131&view=results

    In the meantime I am trying to repro locally.

  6. nohwnd commented on Jan 23, 2024

    @nohwnd
    Member

    I can't repro fully.

    I can run the tests but I always get the full output of the error. And I can't run tests without failures locally because I don't have all the artifacts. I always see: Tarball path '' specified in does not exist. ( from here ) I downloaded what I could but I still cannot get it to work locally.

    I can see in binary logger the same error output as in build, but it is followed by the actual error message.

    In your run I cannot see any failures in trx, so I am not sure where the errors are coming from and if they are real.

    I've edited the target to actually propagate the opt-out flag and started another build here (from a new branch, I want to see how the other one will finish) https://dev.azure.com/dnceng/internal/_build/results?buildId=2360247&view=results

  7. MichaelSimons commented on Feb 1, 2024

    @MichaelSimons
    Member

    [Triage] closing as is appears this fix flowed in.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    ops-monitorIssues created/handled by the source build monitor roleuntriaged

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions