Skip to content

fix: fill the bucket under the limiter's lock - #167

Open
ZacKienzle2 wants to merge 1 commit into
JWCook:mainfrom
ZacKienzle2:fix/fill-bucket-limiter-lock
Open

ZacKienzle2 wants to merge 1 commit into
JWCook:mainfrom
ZacKienzle2:fix/fill-bucket-limiter-lock

Conversation

@ZacKienzle2

Copy link
Copy Markdown

Problem

LimiterMixin._fill_bucket runs after a 429 and loops over bucket.put() without
taking Limiter.lock. In pyrate-limiter 4.5.1, AbstractBucket._record sets
failing_rate on every put, so an admitted filler resets it to None. A second
thread inside Limiter._try_acquire, which does hold the lock, can be between the
None check in AbstractBucket.waiting and _calc_waiting, and fails with:

AssertionError: assert self.failing_rate is not None
pyrate_limiter/abstracts/bucket.py:172

The fallback path that reaches this assert is the one for buckets that record no
wait, so it shows up with a shared PostgresBucket under concurrent use and after
429s.

Fix

Fill inside combined_lock(self.limiter.lock, blocking=True), the same lock
_try_acquire holds. combined_lock handles both forms of Limiter.lock (an
RLock, or a tuple when the bucket has a limiter_lock()). It lives in
pyrate_limiter.limiter and is not in __all__, but it is present across the
supported >=4.1,<5 range.

bucket.now() moves inside the lock so that fillers are not stamped with a time
from before a wait for the lock.

try_acquire(blocking=False) in a loop was rejected: it returns False when the
lock is contended, which would cut the fill short exactly when threads compete.

Test

test_fill_bucket_waits_for_limiter_lock holds the limiter lock, starts
_fill_bucket in a thread, and asserts that nothing reaches the bucket until the
lock is released. It does not depend on the pyrate-limiter version.

_fill_bucket put filler items without taking Limiter.lock. An admitted
filler resets bucket.failing_rate, so a thread between its own put() and
waiting() in try_acquire() could fail with an AssertionError in
AbstractBucket.waiting. The fill now takes that lock, and reads its
timestamp after acquiring it.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant