Repository navigation
Batch 3 — Concurrency (#10 memory locking + CTS race, #11 cache stampede) - #17
Merged
Merged
Conversation
Adds SemaphoreSlimCacheLocker, an in-process keyed mutex filling the TLock seam. Ref-counted entry removal keeps unbounded resource keys (route-templated namespaces) from accumulating. No consumers yet. Claude-Session: https://claude.ai/code/session_01VUPTqPh5w79mfa18fLnMLz
The memory backend passed NullCacheLocker, leaving the TLock seam inert while
the namespace key index was read-modify-written unguarded. With caches created
per request, concurrent writers each rebuilt the index and overwrote one
another: a regression test writing 200 keys through per-request cache instances
retained only 2 of them, silently breaking namespace eviction and refresh for
the rest.
Injects a shared singleton SemaphoreSlimCacheLocker through
MemoryActionCacheFactory and guards the index with it.
Two further fixes fell out of this:
- The index entry was rewritten with the *calling request's* entry options, so
two attributes with different expirations on one namespace made the index's
own expiration flap. The key's absolute expiration and the index entry's
options are now separate parameters.
- The index is now stored under its own "{namespace}:__keys" cache key. It had
been keyed by the Namespace record while ExpirationTokenSources keyed the
namespace's CancellationTokenSource by the equivalent string — distinct only
by accident, and a collision as soon as either side changed key type.
BREAKING: MemoryActionCache now derives from ActionCacheBase<SemaphoreSlimLock>
instead of ActionCacheBase<NullCacheLock>.
BREAKING: MemoryActionCacheFactory's constructor takes a SemaphoreSlimCacheLocker.
Claude-Session: https://claude.ai/code/session_01VUPTqPh5w79mfa18fLnMLz
RemoveAsync() disposed a CancellationTokenSource shared with every in-flight request for the namespace, so a concurrent write reading .Token to build its expiration token threw ObjectDisposedException — a 500 under fail-closed, a silently dropped cache write under fail-open. It also assigned the replacement to the instance field only, never writing it back to the store, so entries written afterwards carried a token no later eviction would ever cancel. Both are reproduced by regression tests that fail against the previous code. Lifecycle moves into ExpirationTokenSources via a new Reset(key) that cancels without disposing: cancellation already releases the registrations and self-evicts the store entry, whose expiration token is that very source. MemoryActionCache no longer holds a mutable source and resolves the live one per operation, which makes the orphaned-token defect structurally impossible rather than merely fixed. BREAKING: IExpirationTokenSources gains Reset(string). BREAKING: MemoryActionCache's constructor takes IExpirationTokenSources in place of CancellationTokenSource. Claude-Session: https://claude.ai/code/session_01VUPTqPh5w79mfa18fLnMLz
Adds IActionCacheSingleFlight and an in-process implementation using lock-then-recheck: the leader executes while waiters block, then re-read the cache and reuse what the leader stored. Verified by a test in which 40 concurrent misses on one key run the value factory exactly once. Lock-acquisition timeouts fall through to uncoalesced execution rather than throwing, matching the library's fail-open stance. Not yet wired into filters. Claude-Session: https://claude.ai/code/session_01VUPTqPh5w79mfa18fLnMLz
Both the MVC and Minimal API cache filters now route their miss branch through IActionCacheSingleFlight, so one hot key's expiry no longer stampedes the origin action. A test issuing 30 concurrent requests against one uncached key executes the action 30 times before this change and once after. On by default; opt out with [ActionCache(SingleFlight = false)], which takes the previous code path verbatim rather than a degenerate lock acquisition. Eviction and refresh filters keep their existing constructors via a null-object, since they never produce cache entries. BREAKING: ActionCacheFilter and ActionCacheEndpointFilter constructors take an IActionCacheSingleFlight and a bool. BREAKING: ActionCacheFilterAbstractFactory and ActionCacheEndpointFilterAbstractFactory constructors take an IActionCacheSingleFlight. BREAKING: IActionCacheFilterAbstractFactory<T> gains a CreateInstance overload. Claude-Session: https://claude.ai/code/session_01VUPTqPh5w79mfa18fLnMLz
options.UseDistributedSingleFlight() coordinates coalescing across instances using the configured Redis or SQL Server distributed lock, giving the fully-implemented but until-now uncalled RedisCacheLocker its first consumer. Redis is preferred when both backends are configured. Throws at registration time when neither backend is present, so the misconfiguration surfaces at startup rather than under load. Lock-acquisition timeouts — which CacheLockerBase signals by throwing — are absorbed and degrade to uncoalesced execution, matching the in-process behavior. Claude-Session: https://claude.ai/code/session_01VUPTqPh5w79mfa18fLnMLz
Twenty concurrent requests to one cached controller action invoke it once; with SingleFlight = false the same test records twenty invocations. Documents stampede protection in the README and corrects CLAUDE.md's stale claim that the memory backend used SemaphoreSlimLock — it used NullCacheLock until this batch. The locking table now covers all four backends. Claude-Session: https://claude.ai/code/session_01VUPTqPh5w79mfa18fLnMLz
This was referenced Aug 21, 2026
…ilure from an exception type Three defects found reviewing stampede protection. 1. The lease was inverted. Single flight borrowed the key-index lock's settings, whose LockDuration (5 s) is shorter than its LockTimeout (10 s). On Redis that duration is the lock key's TTL, so any action taking 5-10 s lost its lock while still running, a waiting caller acquired it, and both executed — precisely the stampede this exists to prevent, and only for the slow actions where it matters. Single flight now has its own ActionCacheSingleFlightOptions (LeaseDuration 30 s, WaitTimeout 10 s), validated at registration so a lease that cannot coalesce anything fails at startup instead of degrading under load. Configure with UseSingleFlightOptions(), or the new overload of UseDistributedSingleFlight(). 2. DistributedSingleFlight could run the action twice. It detected lock timeout by catching InvalidOperationException, which CacheLockerBase throws — but so does the origin action, readily. That logged a misleading lock timeout and then called the value factory a second time; in MVC, a second next() on a context already invoked. ICacheLockerHandler gains TryWaitForLockThenAsync, which reports a busy lock in its result, and exceptions from the work now propagate untouched. 3. LockDuration was honoured by nothing. Only a backend whose locks carry a TTL can impose a lease, and only Redis does — and its cache context uses NullCacheLocker, so the sole real use was the single-flight lock that (1) has now moved off it. Rather than keep a shared option one backend reads, the lease is now a parameter on RedisCacheLocker alone: CacheLockerBase keeps only the wait timeout, and ActionCacheEntryOptions.LockDuration and the write-only CacheLock.Duration are gone. The remaining lock settings say plainly which locker enforces them. BREAKING: ActionCacheEntryOptions.LockDuration and CacheLock.Duration are removed, and the CacheLockerBase, SemaphoreSlimCacheLocker, SqlServerCacheLocker and lock constructors no longer take a lease. RedisCacheLocker keeps one, explicitly.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Third of four themed batches from the recon. Fixes the two concurrency findings, plus two related defects surfaced while doing so.
What was broken
#10a — the memory backend had no locking.
MemoryActionCacheFactorypassedNullCacheLocker, leaving theTLockseam inert while the namespace key index was read-modify-written unguarded. Caches are created per request, so concurrent writers each rebuilt the index and overwrote one another. Measured: a test writing 200 keys through per-request cache instances retained 2 of them — namespace eviction and refresh silently skipped the other 198.#10b —
RemoveAsync()disposed a sharedCancellationTokenSource. Every in-flight request for the namespace holds that instance; once disposed, a concurrent write reading.TokenthrewObjectDisposedException— a 500 under fail-closed, a silently dropped cache write under fail-open. It also assigned the replacement to the instance field only, never writing it back to the store, so entries written afterwards carried a token no later eviction would cancel.#11 — no stampede protection. On expiry of a hot key every concurrent request executed the action. Measured: 30 concurrent requests → 30 executions.
Two defects found on the way:
Namespacerecord whileExpirationTokenSourceskeyed the namespace's CTS by the equivalent string — distinct only by accident, and an outright collision the moment either side changed key type (it did, mid-implementation). The index now has its own{namespace}:__keyskey.What changed
SemaphoreSlimCacheLocker— a ref-counted keyed async mutex. Ref counting matters because route-templated namespaces (Account:{id}) make the key space unbounded. Registered as a singleton; a per-Createlocker guards nothing.ExpirationTokenSourcesviaReset(key), which cancels without disposing. Cancellation already releases registrations and self-evicts the store entry, so not disposing closes theObjectDisposedExceptionwindow entirely.MemoryActionCacheno longer holds a mutable source and resolves the live one per operation — the orphaned-token defect is now structurally impossible rather than merely fixed.IActionCacheSingleFlightusing lock-then-recheck: the leader executes while waiters block, then re-read the cache and reuse what it stored. Chosen over sharing an in-flightTaskbecause that hands one request's result object to another before it has been through the cache, and has no natural distributed form. On by default;[ActionCache(SingleFlight = false)]takes the previous code path verbatim.options.UseDistributedSingleFlight()— opt-in, coalesces across instances via the Redis or SQL Server lock. Gives the fully-implemented but until-now uncalledRedisCacheLockerits first consumer. Throws at startup when neither backend is configured.Lock-acquisition timeouts never throw: they degrade to uncoalesced execution, consistent with the fail-open stance from Batch 2.
Breaking changes (pre-1.0)
MemoryActionCachederives fromActionCacheBase<SemaphoreSlimLock>(wasActionCacheBase<NullCacheLock>).MemoryActionCacheconstructor takesIExpirationTokenSourcesin place ofCancellationTokenSource.MemoryActionCacheFactoryconstructor takes aSemaphoreSlimCacheLocker.IExpirationTokenSourcesgainsReset(string).ActionCacheFilter/ActionCacheEndpointFilterconstructors take anIActionCacheSingleFlightand abool.ActionCacheFilterAbstractFactory/ActionCacheEndpointFilterAbstractFactoryconstructors take anIActionCacheSingleFlight.IActionCacheFilterAbstractFactory<T>gains aCreateInstanceoverload (additive; existing overloads delegate withsingleFlight: true).Eviction and refresh filters keep their existing constructors via a null-object — they never produce cache entries.
Verification
net8.0andnet10.0.ObjectDisposedExceptionraised, orphaned entry surviving eviction, 30 executions instead of 1, and 20 concurrent HTTP requests instead of 1.One note: on the first full integration run, the pre-existing
Test_GetAsync_Expires(5s TTL, 10s sleep) failed once onnet8.0under load. It passed in isolation and on two subsequent full runs. Timing-sensitive and unrelated to these changes, but flagging it rather than burying it.Not in this batch
The layered-cache defects (
ActionCacheHandler.GetKeysAsync's dead fallback, no L1 backfill on an L2 hit) andCancellationTokenonIActionCacheare Batch 5. The single-flight interface is shaped so adding a token later is additive.https://claude.ai/code/session_01VUPTqPh5w79mfa18fLnMLz