Skip to content

requireSnapshotBeforeProcessing#2152

Open
shultseva wants to merge 3 commits intohazelcast:mainfrom
shultseva:CTT-996_initial_snapshot
Open

requireSnapshotBeforeProcessing#2152
shultseva wants to merge 3 commits intohazelcast:mainfrom
shultseva:CTT-996_initial_snapshot

Conversation

@shultseva
Copy link
Copy Markdown
Contributor

@shultseva shultseva commented Mar 30, 2026

@shultseva shultseva requested a review from a team as a code owner March 30, 2026 13:52
@netlify
Copy link
Copy Markdown

netlify bot commented Mar 30, 2026

Deploy Preview for hardcore-allen-f5257d ready!

Name Link
🔨 Latest commit 98d8ceb
🔍 Latest deploy log https://app.netlify.com/projects/hardcore-allen-f5257d/deploys/69ce742af2c9e200070a4cf5
😎 Deploy Preview https://deploy-preview-2152--hardcore-allen-f5257d.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@shultseva shultseva added this to the 5.7.0 milestone Mar 30, 2026
Afterwards, the regular fault tolerance mechanism described above is used.
This option is not supported when processing guarantees are disabled.

Kafka source is a stateful source—it remembers the last processed offset.
Copy link
Copy Markdown
Contributor

@k-jamroz k-jamroz Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So to ensure no data loss, enable setRequireSnapshotBeforeProcessing

this is not always necessary

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

partition count of `271` and journal size of `100,000` the journal only
has space for `370` events per partitions.

Map journal source is a stateful source—it remembers the last processed offset.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So to ensure no data loss

this is not always necessary, eg. with START_FROM_OLDEST, or it might apply in this case only when someone relies on events lost flag.

the docs should be more nuanced

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shultseva and others added 2 commits April 1, 2026 10:02
Co-authored-by: Krzysztof Jamróz <79092062+k-jamroz@users.noreply.github.com>
@shultseva shultseva requested a review from k-jamroz April 2, 2026 13:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants