Repository navigation
Add producer-side ack channel to shared-stream triggers #67523
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
15 commits
Select commit
Hold shift + click to select a range
9830478
feat(triggers): add producer-side ack channel to shared-stream triggers
Lee-W 84af8fb
feat(triggers): dispatch shared-stream broker advances in per-lane order
Lee-W f61c14f
docs(triggers): drop redundant :py prefixes from Sphinx roles in even…
Lee-W d547cd2
refactor(triggers): drop redundant comment on TYPE_CHECKING import
Lee-W 960a866
fix(triggers): gate shared-stream broker advance on trigger event per…
Lee-W f1b35bb
docs(triggers): note disabling Kafka auto-commit in the shared-stream…
Lee-W c20c53e
refactor(triggers): replace triggerer event tuples with a named tuple
Lee-W be51a76
refactor(triggers): compute shared-stream ack mode once at group cons…
Lee-W 0df647e
refactor(triggers): consolidate shared-stream ack tests with parametrize
Lee-W 3a30e9e
refactor(triggers): drop public ack tokens from shared-stream ack mode
Lee-W e45b451
fix(triggers): redeliver never-pulled shared-stream events on unsubsc…
Lee-W 92c2801
test(triggers): drop unused session fixture from shared-stream ack in…
Lee-W 3cf823f
docs(triggers): route non-clean outcomes to a DLQ in shared-stream ad…
Lee-W a393fed
improvement(triggers): log shared-stream event rejection for observab…
Lee-W e1c9069
fix(triggers): terminate shared-stream group when producer advance fails
Lee-W File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
296 changes: 280 additions & 16 deletions
296
airflow-core/docs/authoring-and-scheduling/event-scheduling.rst
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| Shared-stream triggers now support a producer-side ack channel so message-broker upstreams (Kafka, SQS, Pub/Sub, Service Bus) can commit, delete, or ack on the broker handle after all subscribers have accepted an event and the resulting trigger events have been persisted to the metadata database. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.