Skip to content

Fix Allure wiring: remove duplicate attributes and fix CI check for transitive dependencies#1784

Open
vazois wants to merge 5 commits intodevfrom
vazois/ci-runtime
Open

Fix Allure wiring: remove duplicate attributes and fix CI check for transitive dependencies#1784
vazois wants to merge 5 commits intodevfrom
vazois/ci-runtime

Conversation

@vazois
Copy link
Copy Markdown
Contributor

@vazois vazois commented May 9, 2026 •
edited
Loading

Summary

Fixes two CI issues with Allure test wiring in the newly split cluster replication test projects.

Issue 1 - Runtime Allure error

Derived test classes (TLS, AsyncReplay, MultiLog) had [AllureNUnit] applied directly and inherited it from their base classes. NUnit discovers ITestAction attributes from both, causing Allure's BeforeTest to run twice and crash with:

System.InvalidOperationException: Unable to change the container context because the test context is active.

Fix: Removed the duplicate [AllureNUnit] from the 4 derived classes - the inherited attribute from the base class is sufficient.

Issue 2 - CI Allure wiring check failure

The CI Verify Allure wiring step used Assembly.Load() to find AllureTestBase in referenced assemblies. This failed because:

  1. Assembly.Load() does not probe the build output directory (only works for GAC/default paths)
  2. AllureTestBase lives in Garnet.test.cluster, which is a transitive dependency for some projects (e.g., TLS references it through Garnet.test.cluster.replication, not directly)

Fix: Updated the CI script to search AppDomain.GetAssemblies() - all transitive dependencies are already loaded by the runtime during GetTypes().

Changes

  • ClusterReplicationTLS.cs - remove duplicate [AllureNUnit]
  • ClusterReplicationAsyncReplay.cs - remove duplicate [AllureNUnit]
  • ClusterReplicationShardedLog.cs - remove duplicate [AllureNUnit]
  • ClusterReplicationDisklessSyncShardedLog.cs - remove duplicate [AllureNUnit]
  • .github/workflows/ci.yml - fix AllureTestBase resolution for transitive dependencies

vazois and others added 5 commits May 8, 2026 16:55
@vazois
Split cluster tests into 5 parallel CI projects
914f8b1 
Split Garnet.test.cluster into separate projects to enable parallel CI:
- Garnet.test.cluster: shared infra + basic cluster tests (144 tests)
- Garnet.test.cluster.migrate: migrate + slot verification (67 tests)
- Garnet.test.cluster.replication: all replication tests (372 tests)
- Garnet.test.cluster.vectorsets: vector set cluster tests (30 tests)
- Garnet.test.cluster.multilog: sharded log replication tests (163 tests)

Child projects reference base via ProjectReference + InternalsVisibleTo.
Updated CI and nightly workflow matrices to run all 5 in parallel.
Fixed Allure wiring check to find AllureTestBase in referenced assemblies.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@vazois
Split replication tests into TLS, AsyncReplay, and DisklessSync projects
383f40a 
Separate Garnet.test.cluster.replication into 3 additional projects
for parallel CI execution:

- Garnet.test.cluster.replication.tls: TLS replication tests
  (inherits ClusterReplicationBaseTests with useTLS=true)
- Garnet.test.cluster.replication.asyncreplay: Async replay tests
  (inherits ClusterReplicationBaseTests with asyncReplay=true)
- Garnet.test.cluster.replication.disklesssync: Diskless sync tests
  (standalone ClusterReplicationDisklessSyncTests)

Base replication project retains ClusterReplicationBaseTests and
ClusterResetDuringReplicationTests. Updated InternalsVisibleTo,
multilog project references, CI and nightly workflow matrices,
and solution file.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@vazois
Split Garnet.test into 8 projects for parallel CI execution
faf3e97 
Split the monolithic Garnet.test project into 8 focused test projects:
- Garnet.test (base): RESP core, config, admin, infra (~774 tests)
- Garnet.test.collections: Hash, List, Set, SortedSet, Geo (~746 tests)
- Garnet.test.acl: ACL and auth tests (~426 tests)
- Garnet.test.scripting: Lua, custom commands, transactions, AOF, modules (~585 tests)
- Garnet.test.complexstring: Bitmap, HyperLogLog (~386 tests)
- Garnet.test.vectorset: VectorSet tests (~36 tests)
- Garnet.test.rangeindex: RangeIndex tests (~58 tests)
- Garnet.test.extensions: JSON, DiskANN, revivification, storage internals (~527 tests)

Each child project references Garnet.test for shared infrastructure (TestUtils,
AllureTestBase, extensions). InternalsVisibleTo entries added to Garnet.server,
Garnet.host, GarnetServer, GarnetJSON, and Tsavorite.core for child projects.

Updated CI and nightly workflows with expanded test matrices.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@vazois
Add missing [AllureNUnit] attributes to cluster sub-project test fixt…
39b0aad 
…ures

Added [AllureNUnit] and [TestFixture] to ClusterReplicationTLS,
ClusterReplicationAsyncReplay, ClusterReplicationShardedLog, and
ClusterReplicationDisklessSyncShardedLog to pass CI Allure wiring check.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@vazois
Fix Allure wiring: remove duplicate [AllureNUnit] and fix CI check fo…
ded578a 
…r transitive dependencies

- Remove duplicate [AllureNUnit] from derived test classes (TLS, AsyncReplay,
  MultiLog) that inherit it from base classes, fixing runtime error
  'Unable to change the container context because the test context is active'
- Fix CI Allure wiring check to search AppDomain.GetAssemblies() instead of
  using Assembly.Load() on direct references, which failed for AllureTestBase
  in transitive dependencies (e.g. Garnet.test.cluster via Garnet.test.cluster.replication)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings May 9, 2026 04:35
Copilot AI reviewed May 9, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR addresses Allure/NUnit wiring issues after splitting tests into multiple projects, and updates CI to correctly validate Allure inheritance across transitive dependencies. It also introduces several new split test projects and adds/relocates a number of test suites into those projects.

Changes:

  • Remove duplicate Allure attributes from derived fixtures and ensure derived fixtures are still discovered by NUnit.
  • Update CI “Verify Allure wiring” to resolve AllureTestBase from transitive dependencies via loaded AppDomain assemblies.
  • Split tests into additional projects (collections/ACL/scripting/complexstring/vectorset/rangeindex/extensions + cluster sub-suites) and update InternalsVisibleTo, solution, and workflow matrices accordingly.

Reviewed changes

Copilot reviewed 29 out of 91 changed files in this pull request and generated no comments.

Show a summary per file
File Description
test/Garnet.test/Garnet.test.csproj Adds InternalsVisibleTo entries to support newly split test projects; removes linked global usings include.
test/Garnet.test.vectorset/VectorCleanupVsResetRaceTests.cs Adds regression test for VectorManager cleanup vs store reset race using UnsafeAccessor.
test/Garnet.test.vectorset/Garnet.test.vectorset.csproj New vector-set focused test project.
test/Garnet.test.scripting/Garnet.test.scripting.csproj New scripting-focused test project.
test/Garnet.test.rangeindex/Garnet.test.rangeindex.csproj New range-index focused test project.
test/Garnet.test.extensions/ScratchBufferAllocatorTests.cs Adds tests for ScratchBufferAllocator behavior.
test/Garnet.test.extensions/RespRevivificationTests.cs Adds revivification behavior tests (RMW/Upsert/AOF interactions).
test/Garnet.test.extensions/ReadOptimizedLockTests.cs Adds unit tests for ReadOptimizedLock correctness under contention.
test/Garnet.test.extensions/ReadCacheTests.cs Adds tests validating read-cache population/eviction for string and object records.
test/Garnet.test.extensions/IndexGrowthTests.cs Adds tests validating index resizing behavior (including checkpoint/recovery).
test/Garnet.test.extensions/GlobalUsings.cs Provides local aliases used by extensions tests.
test/Garnet.test.extensions/GarnetJSON/JSONPath/QueryExpressionTests.cs Adds JSONPath query expression tests under extensions suite.
test/Garnet.test.extensions/GarnetJSON/JSONPath/JsonPathRegressionTests.cs Adds JSONPath regression suite runner using embedded JSON cases.
test/Garnet.test.extensions/GarnetJSON/JSONPath/JsonAssert.cs Adds JSON element/node equality helpers for JSONPath tests.
test/Garnet.test.extensions/Garnet.test.extensions.csproj New extensions test project (includes GarnetJSON + DiskANN tests/resources).
test/Garnet.test.extensions/DiskANN/DiskANNSyntheticRecallTests.cs Adds DiskANN synthetic recall tests (grid/circle datasets).
test/Garnet.test.complexstring/Garnet.test.complexstring.csproj New complex-string focused test project.
test/Garnet.test.collections/RespListGarnetClientTests.cs Adds/relocates GarnetClient list command tests into collections suite.
test/Garnet.test.collections/GeoHashTests.cs Adds GeoHash encode/decode tests into collections suite.
test/Garnet.test.collections/GarnetObjectTests.cs Adds object serialization/checkpoint/recovery tests using Tsavorite directly.
test/Garnet.test.collections/Garnet.test.collections.csproj New collections-focused test project (links shared global usings).
test/Garnet.test.cluster/Garnet.test.cluster.csproj Adds InternalsVisibleTo for new cluster sub-projects; links AllureTestBase.
test/Garnet.test.cluster/CustomProcs/ClusterDelRmw.cs Makes ClusterDelRmw public for cross-assembly use.
test/Garnet.test.cluster.vectorsets/Garnet.test.cluster.vectorsets.csproj New cluster vector-sets test project.
test/Garnet.test.cluster.replication/ReplicationTests/ClusterResetDuringReplicationTests.cs Adds DEBUG-only tests for CLUSTER RESET HARD during replication attach.
test/Garnet.test.cluster.replication/Garnet.test.cluster.replication.csproj New cluster replication test project; exposes internals to dependent suites.
test/Garnet.test.cluster.replication.tls/ReplicationTests/ClusterReplicationTLS.cs Removes duplicate Allure attribute by relying on base; ensures fixture discovery.
test/Garnet.test.cluster.replication.tls/Garnet.test.cluster.replication.tls.csproj New TLS-specific replication test project.
test/Garnet.test.cluster.replication.disklesssync/Garnet.test.cluster.replication.disklesssync.csproj New diskless-sync focused replication test project.
test/Garnet.test.cluster.replication.asyncreplay/ReplicationTests/ClusterReplicationAsyncReplay.cs Removes duplicate Allure attribute by relying on base; ensures fixture discovery.
test/Garnet.test.cluster.replication.asyncreplay/Garnet.test.cluster.replication.asyncreplay.csproj New async-replay replication test project.
test/Garnet.test.cluster.multilog/MultiLogTests/ClusterReplicationShardedLog.cs Adds multilog/sharded-log replication test fixture wrapper.
test/Garnet.test.cluster.multilog/MultiLogTests/ClusterReplicationDisklessSyncShardedLog.cs Adds multilog wrapper for diskless-sync replication tests.
test/Garnet.test.cluster.multilog/Garnet.test.cluster.multilog.csproj New multilog-focused cluster test project.
test/Garnet.test.cluster.migrate/RedirectTests/TestClusterProc.cs Adds custom txn procedures and redirect test commands for migrate suite.
test/Garnet.test.cluster.migrate/Garnet.test.cluster.migrate.csproj New cluster migrate test project.
test/Garnet.test.cluster.migrate/ClusterMigrateTLSTests.cs Adds TLS variant wrapper fixture for cluster migrate tests.
test/Garnet.test.acl/Resp/GarnetAuthenticatorTests.cs Adds tests for authenticator invalidation behavior.
test/Garnet.test.acl/Resp/ACL/UserAclResult.cs Adds helper for parsing ACL GETUSER responses.
test/Garnet.test.acl/Resp/ACL/ParallelTests.cs Adds parallel stress tests for AUTH/password hashing/ACL SETUSER.
test/Garnet.test.acl/Resp/ACL/GetUserTests.cs Adds ACL GETUSER tests including multi-user scenarios.
test/Garnet.test.acl/Resp/ACL/DeleteUserTests.cs Adds ACL DELUSER behavior tests.
test/Garnet.test.acl/Resp/ACL/BasicTests.cs Adds tests for basic ACL commands and NoAuth flag validation.
test/Garnet.test.acl/Resp/ACL/AclTest.cs Introduces ACL test base class with shared setup/teardown and constants.
test/Garnet.test.acl/Resp/ACL/AclParserTests.cs Adds ACL parser reduction/timeout regression tests.
test/Garnet.test.acl/Resp/ACL/AclConfigurationFileTests.cs Adds tests around ACL file load/save semantics and error handling.
test/Garnet.test.acl/Garnet.test.acl.csproj New ACL-focused test project.
playground/TstRunner/TstRunner.csproj Adds references to new cluster migrate/replication test projects.
modules/GarnetJSON/GarnetJSON.csproj Adds InternalsVisibleTo for Garnet.test.extensions.
main/GarnetServer/GarnetServer.csproj Adds InternalsVisibleTo for newly split test projects.
libs/storage/Tsavorite/cs/src/core/Tsavorite.core.csproj Adds InternalsVisibleTo for Garnet.test.extensions.
libs/server/Properties/AssemblyInfo.cs Adds InternalsVisibleTo for newly split test projects.
libs/host/Garnet.host.csproj Adds InternalsVisibleTo for newly split test projects.
Garnet.slnx Adds new test projects to the solution.
.github/workflows/nightly.yml Expands nightly test matrix to include new test projects.
.github/workflows/ci.yml Expands CI test matrix and fixes Allure wiring verification for transitive deps.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments

@badrishc badrishc Awaiting requested review from badrishc badrishc is a code owner

@TedHartMS TedHartMS Awaiting requested review from TedHartMS TedHartMS is a code owner

@kevin-montrose kevin-montrose Awaiting requested review from kevin-montrose kevin-montrose is a code owner

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@vazois