feat(async): Set priorities to async tasks #14195
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
the
settings_changes
kiblik
force-pushed
the
celery_set_prio
branch
3 times, most recently
from
95561b9 to
1f8e468
Compare
Signed-off-by: kiblik <[email protected]>
Co-authored-by: valentijnscholten <[email protected]>
kiblik
force-pushed
the
celery_set_prio
branch
from
53a8e1e to
e882d0f
Compare
kiblik
force-pushed
the
celery_set_prio
branch
2 times, most recently
from
3dd4cdb to
6eb4791
Compare
Signed-off-by: kiblik <[email protected]>
kiblik
force-pushed
the
celery_set_prio
branch
from
6eb4791 to
161b919
Compare
valentijnscholten
added
the
affects_pro
Member
|
I've converted it to draft as we can only test/merge this after 2.55.2 has been released and merged back into |
Contributor
|
This pull request has conflicts, please resolve those before we can evaluate the pull request. |
Resolved conflicts by: - Removing @dojo_async_task decorators that were removed in upstream - Keeping priority parameters from the PR - Adapting to refactored notification system (standalone task functions) - Using correct priority values (3 for notifications/jira, 1 for webhooks, 4 for cleanup tasks, 0 for status checks)
Contributor
|
Conflicts have been resolved. A maintainer will review the pull request shortly. |
valentijnscholten
removed
the
affects_pro
Member
|
@kiblik I resolved conflicts and published the PR so we can merge it and test it with Pro. The PR does not affect Pro in the sense that Pro will keep working and the Pro celery tasks will get priority 3 which is fine. But we can/should do a Pro PR together or after this one to move some of the Pro celery tasks to other priority levels. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR is adding support for priority queues. More important tasks will be processed sooner and vice versa.
The status of each queue is listed in the system settings overview:
Queue with id
0is automatically marked without a suffix (all other are using naming conventioncelery:1,celery:2...). Thanks to this fact, we are keeping backward compatibility (if there are any tasks in the queue during migration to the new version, we will still process those tasks after the upgrade).It is a bit harder to write tests for this. But I tested manually (stop worker, play around, start worker, observe logs) and it was working as expected.
The role of each queue is described in
settings.dist.py(and reflected in the overview in system settings). If task do not have a priority assigned, priority is set to the default value:3I'm open to feedback if priority of any task should be changed.