Skip to content

Handle GitHub API rate limit errors appropriately#4299

Merged
HebaruSan merged 1 commit into
KSP-CKAN:masterfrom
HebaruSan:fix/sd-gh-throttling
Feb 6, 2025
Merged

Handle GitHub API rate limit errors appropriately#4299
HebaruSan merged 1 commit into
KSP-CKAN:masterfrom
HebaruSan:fix/sd-gh-throttling

Conversation

@HebaruSan
Copy link
Copy Markdown
Member

@HebaruSan HebaruSan commented Feb 6, 2025

Motivation

The bot has occasionally hit the GitHub API rate limit recently. When this happens, it keeps querying over and over regardless:

image

... which isn't what we're supposed to do:

https://docs.github.com/en/rest/using-the-rest-api/best-practices-for-using-the-rest-api#handle-rate-limit-errors-appropriately

image

Changes

Now if we get a rate limiting response from the GitHub API, we pause inflation for the time it requests and log that time span in the current mod's inflation error. This should allow the rate limit budget to recharge and keep us safer from being banned.

Known limitations

Due to how the logging works, the messages about pausing will only appear after the pause occurs. The Discord messages are sent by the Infra Python code, which receives them from AWS queue messages, which are batched in groups of 10. So if the third mod of a batch of 10 hits the limit, we'll pause for however long, then continue with mods 4–10 and finally send the message that alerts us to the pause.

I looked for a way around this, but did not find one. Maybe we can figure it out in the future.

@HebaruSan HebaruSan added Bug Something is not working as intended Enhancement New features or functionality Core (ckan.dll) Issues affecting the core part of CKAN Netkan Issues affecting the netkan data Network Issues affecting internet connections of CKAN labels Feb 6, 2025
@HebaruSan HebaruSan merged commit 1d05ec5 into KSP-CKAN:master Feb 6, 2025
@HebaruSan HebaruSan deleted the fix/sd-gh-throttling branch February 6, 2025 16:49
@HebaruSan
Copy link
Copy Markdown
Member Author

So far so good:

image

🤞

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Bug Something is not working as intended Core (ckan.dll) Issues affecting the core part of CKAN Enhancement New features or functionality Netkan Issues affecting the netkan data Network Issues affecting internet connections of CKAN

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant