Skip to content

Batch 3 — Concurrency (#10 memory locking + CTS race, #11 cache stampede) - #17

Merged
jzills merged 8 commits into
developfrom
fix/batch-3-concurrency
Aug 23, 2026
Merged

jzills merged 8 commits into
developfrom
fix/batch-3-concurrency

Conversation

@jzills

@jzills jzills commented Aug 21, 2026

Copy link
Copy Markdown
Owner

Third of four themed batches from the recon. Fixes the two concurrency findings, plus two related defects surfaced while doing so.

What was broken

#10a — the memory backend had no locking. MemoryActionCacheFactory passed NullCacheLocker, leaving the TLock seam inert while the namespace key index was read-modify-written unguarded. Caches are created per request, so concurrent writers each rebuilt the index and overwrote one another. Measured: a test writing 200 keys through per-request cache instances retained 2 of them — namespace eviction and refresh silently skipped the other 198.

#10b — RemoveAsync() disposed a shared CancellationTokenSource. Every in-flight request for the namespace holds that instance; once disposed, a concurrent write reading .Token threw ObjectDisposedException — a 500 under fail-closed, a silently dropped cache write under fail-open. It also assigned the replacement to the instance field only, never writing it back to the store, so entries written afterwards carried a token no later eviction would cancel.

#11 — no stampede protection. On expiry of a hot key every concurrent request executed the action. Measured: 30 concurrent requests → 30 executions.

Two defects found on the way:

  • The index entry was rewritten with the calling request's entry options, so two attributes with different expirations on one namespace made the index's own expiration flap.
  • The index was keyed by the Namespace record while ExpirationTokenSources keyed the namespace's CTS by the equivalent string — distinct only by accident, and an outright collision the moment either side changed key type (it did, mid-implementation). The index now has its own {namespace}:__keys key.

What changed

  • SemaphoreSlimCacheLocker — a ref-counted keyed async mutex. Ref counting matters because route-templated namespaces (Account:{id}) make the key space unbounded. Registered as a singleton; a per-Create locker guards nothing.
  • CTS lifecycle moved into ExpirationTokenSources via Reset(key), which cancels without disposing. Cancellation already releases registrations and self-evicts the store entry, so not disposing closes the ObjectDisposedException window entirely. MemoryActionCache no longer holds a mutable source and resolves the live one per operation — the orphaned-token defect is now structurally impossible rather than merely fixed.
  • IActionCacheSingleFlight using lock-then-recheck: the leader executes while waiters block, then re-read the cache and reuse what it stored. Chosen over sharing an in-flight Task because that hands one request's result object to another before it has been through the cache, and has no natural distributed form. On by default; [ActionCache(SingleFlight = false)] takes the previous code path verbatim.
  • options.UseDistributedSingleFlight() — opt-in, coalesces across instances via the Redis or SQL Server lock. Gives the fully-implemented but until-now uncalled RedisCacheLocker its first consumer. Throws at startup when neither backend is configured.

Lock-acquisition timeouts never throw: they degrade to uncoalesced execution, consistent with the fail-open stance from Batch 2.

Breaking changes (pre-1.0)

  • MemoryActionCache derives from ActionCacheBase<SemaphoreSlimLock> (was ActionCacheBase<NullCacheLock>).
  • MemoryActionCache constructor takes IExpirationTokenSources in place of CancellationTokenSource.
  • MemoryActionCacheFactory constructor takes a SemaphoreSlimCacheLocker.
  • IExpirationTokenSources gains Reset(string).
  • ActionCacheFilter / ActionCacheEndpointFilter constructors take an IActionCacheSingleFlight and a bool.
  • ActionCacheFilterAbstractFactory / ActionCacheEndpointFilterAbstractFactory constructors take an IActionCacheSingleFlight.
  • IActionCacheFilterAbstractFactory<T> gains a CreateInstance overload (additive; existing overloads delegate with singleFlight: true).

Eviction and refresh filters keep their existing constructors via a null-object — they never produce cache entries.

Verification

  • Unit: 473/473 on net8.0 and net10.0.
  • Integration: 47/47 on both TFMs, against live Redis/SQL Server/Cosmos.
  • Release build: 0 warnings, 0 errors.
  • Every regression test was confirmed to fail against the pre-change code: 200 keys → 2, ObjectDisposedException raised, orphaned entry surviving eviction, 30 executions instead of 1, and 20 concurrent HTTP requests instead of 1.

One note: on the first full integration run, the pre-existing Test_GetAsync_Expires (5s TTL, 10s sleep) failed once on net8.0 under load. It passed in isolation and on two subsequent full runs. Timing-sensitive and unrelated to these changes, but flagging it rather than burying it.

Not in this batch

The layered-cache defects (ActionCacheHandler.GetKeysAsync's dead fallback, no L1 backfill on an L2 hit) and CancellationToken on IActionCache are Batch 5. The single-flight interface is shaped so adding a token later is additive.

https://claude.ai/code/session_01VUPTqPh5w79mfa18fLnMLz

jzills added 7 commits August 21, 2026 14:35
Adds SemaphoreSlimCacheLocker, an in-process keyed mutex filling the TLock
seam. Ref-counted entry removal keeps unbounded resource keys (route-templated
namespaces) from accumulating. No consumers yet.

Claude-Session: https://claude.ai/code/session_01VUPTqPh5w79mfa18fLnMLz
The memory backend passed NullCacheLocker, leaving the TLock seam inert while
the namespace key index was read-modify-written unguarded. With caches created
per request, concurrent writers each rebuilt the index and overwrote one
another: a regression test writing 200 keys through per-request cache instances
retained only 2 of them, silently breaking namespace eviction and refresh for
the rest.

Injects a shared singleton SemaphoreSlimCacheLocker through
MemoryActionCacheFactory and guards the index with it.

Two further fixes fell out of this:
- The index entry was rewritten with the *calling request's* entry options, so
  two attributes with different expirations on one namespace made the index's
  own expiration flap. The key's absolute expiration and the index entry's
  options are now separate parameters.
- The index is now stored under its own "{namespace}:__keys" cache key. It had
  been keyed by the Namespace record while ExpirationTokenSources keyed the
  namespace's CancellationTokenSource by the equivalent string — distinct only
  by accident, and a collision as soon as either side changed key type.

BREAKING: MemoryActionCache now derives from ActionCacheBase<SemaphoreSlimLock>
instead of ActionCacheBase<NullCacheLock>.
BREAKING: MemoryActionCacheFactory's constructor takes a SemaphoreSlimCacheLocker.

Claude-Session: https://claude.ai/code/session_01VUPTqPh5w79mfa18fLnMLz
RemoveAsync() disposed a CancellationTokenSource shared with every in-flight
request for the namespace, so a concurrent write reading .Token to build its
expiration token threw ObjectDisposedException — a 500 under fail-closed, a
silently dropped cache write under fail-open. It also assigned the replacement
to the instance field only, never writing it back to the store, so entries
written afterwards carried a token no later eviction would ever cancel.

Both are reproduced by regression tests that fail against the previous code.

Lifecycle moves into ExpirationTokenSources via a new Reset(key) that cancels
without disposing: cancellation already releases the registrations and
self-evicts the store entry, whose expiration token is that very source.
MemoryActionCache no longer holds a mutable source and resolves the live one
per operation, which makes the orphaned-token defect structurally impossible
rather than merely fixed.

BREAKING: IExpirationTokenSources gains Reset(string).
BREAKING: MemoryActionCache's constructor takes IExpirationTokenSources in
place of CancellationTokenSource.

Claude-Session: https://claude.ai/code/session_01VUPTqPh5w79mfa18fLnMLz
Adds IActionCacheSingleFlight and an in-process implementation using
lock-then-recheck: the leader executes while waiters block, then re-read the
cache and reuse what the leader stored. Verified by a test in which 40
concurrent misses on one key run the value factory exactly once.

Lock-acquisition timeouts fall through to uncoalesced execution rather than
throwing, matching the library's fail-open stance. Not yet wired into filters.

Claude-Session: https://claude.ai/code/session_01VUPTqPh5w79mfa18fLnMLz
Both the MVC and Minimal API cache filters now route their miss branch through
IActionCacheSingleFlight, so one hot key's expiry no longer stampedes the origin
action. A test issuing 30 concurrent requests against one uncached key executes
the action 30 times before this change and once after.

On by default; opt out with [ActionCache(SingleFlight = false)], which takes the
previous code path verbatim rather than a degenerate lock acquisition. Eviction
and refresh filters keep their existing constructors via a null-object, since
they never produce cache entries.

BREAKING: ActionCacheFilter and ActionCacheEndpointFilter constructors take an
IActionCacheSingleFlight and a bool.
BREAKING: ActionCacheFilterAbstractFactory and
ActionCacheEndpointFilterAbstractFactory constructors take an
IActionCacheSingleFlight.
BREAKING: IActionCacheFilterAbstractFactory<T> gains a CreateInstance overload.

Claude-Session: https://claude.ai/code/session_01VUPTqPh5w79mfa18fLnMLz
options.UseDistributedSingleFlight() coordinates coalescing across instances
using the configured Redis or SQL Server distributed lock, giving the
fully-implemented but until-now uncalled RedisCacheLocker its first consumer.
Redis is preferred when both backends are configured.

Throws at registration time when neither backend is present, so the
misconfiguration surfaces at startup rather than under load. Lock-acquisition
timeouts — which CacheLockerBase signals by throwing — are absorbed and degrade
to uncoalesced execution, matching the in-process behavior.

Claude-Session: https://claude.ai/code/session_01VUPTqPh5w79mfa18fLnMLz
Twenty concurrent requests to one cached controller action invoke it once; with
SingleFlight = false the same test records twenty invocations.

Documents stampede protection in the README and corrects CLAUDE.md's stale
claim that the memory backend used SemaphoreSlimLock — it used NullCacheLock
until this batch. The locking table now covers all four backends.

Claude-Session: https://claude.ai/code/session_01VUPTqPh5w79mfa18fLnMLz
…ilure from an exception type

Three defects found reviewing stampede protection.

1. The lease was inverted. Single flight borrowed the key-index lock's settings,
whose LockDuration (5 s) is shorter than its LockTimeout (10 s). On Redis that
duration is the lock key's TTL, so any action taking 5-10 s lost its lock while
still running, a waiting caller acquired it, and both executed — precisely the
stampede this exists to prevent, and only for the slow actions where it matters.
Single flight now has its own ActionCacheSingleFlightOptions (LeaseDuration 30 s,
WaitTimeout 10 s), validated at registration so a lease that cannot coalesce
anything fails at startup instead of degrading under load. Configure with
UseSingleFlightOptions(), or the new overload of UseDistributedSingleFlight().

2. DistributedSingleFlight could run the action twice. It detected lock timeout
by catching InvalidOperationException, which CacheLockerBase throws — but so does
the origin action, readily. That logged a misleading lock timeout and then called
the value factory a second time; in MVC, a second next() on a context already
invoked. ICacheLockerHandler gains TryWaitForLockThenAsync, which reports a busy
lock in its result, and exceptions from the work now propagate untouched.

3. LockDuration was honoured by nothing. Only a backend whose locks carry a TTL
can impose a lease, and only Redis does — and its cache context uses NullCacheLocker,
so the sole real use was the single-flight lock that (1) has now moved off it.
Rather than keep a shared option one backend reads, the lease is now a parameter
on RedisCacheLocker alone: CacheLockerBase keeps only the wait timeout, and
ActionCacheEntryOptions.LockDuration and the write-only CacheLock.Duration are
gone. The remaining lock settings say plainly which locker enforces them.

BREAKING: ActionCacheEntryOptions.LockDuration and CacheLock.Duration are removed,
and the CacheLockerBase, SemaphoreSlimCacheLocker, SqlServerCacheLocker and lock
constructors no longer take a lease. RedisCacheLocker keeps one, explicitly.
@jzills
jzills merged commit 7c8ec6f into develop Aug 23, 2026
4 checks passed
@jzills jzills mentioned this pull request Aug 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant