Repository navigation
fix: fill the bucket under the limiter's lock - #167
Open
ZacKienzle2 wants to merge 1 commit into
Open
ZacKienzle2 wants to merge 1 commit into
ZacKienzle2 wants to merge 1 commit into
Conversation
_fill_bucket put filler items without taking Limiter.lock. An admitted filler resets bucket.failing_rate, so a thread between its own put() and waiting() in try_acquire() could fail with an AssertionError in AbstractBucket.waiting. The fill now takes that lock, and reads its timestamp after acquiring it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
LimiterMixin._fill_bucketruns after a 429 and loops overbucket.put()withouttaking
Limiter.lock. In pyrate-limiter 4.5.1,AbstractBucket._recordsetsfailing_rateon every put, so an admitted filler resets it toNone. A secondthread inside
Limiter._try_acquire, which does hold the lock, can be between theNonecheck inAbstractBucket.waitingand_calc_waiting, and fails with:The fallback path that reaches this assert is the one for buckets that record no
wait, so it shows up with a shared
PostgresBucketunder concurrent use and after429s.
Fix
Fill inside
combined_lock(self.limiter.lock, blocking=True), the same lock_try_acquireholds.combined_lockhandles both forms ofLimiter.lock(anRLock, or a tuple when the bucket has alimiter_lock()). It lives inpyrate_limiter.limiterand is not in__all__, but it is present across thesupported
>=4.1,<5range.bucket.now()moves inside the lock so that fillers are not stamped with a timefrom before a wait for the lock.
try_acquire(blocking=False)in a loop was rejected: it returnsFalsewhen thelock is contended, which would cut the fill short exactly when threads compete.
Test
test_fill_bucket_waits_for_limiter_lockholds the limiter lock, starts_fill_bucketin a thread, and asserts that nothing reaches the bucket until thelock is released. It does not depend on the pyrate-limiter version.