Skip to content

Fix ParallelStreamingDataset with resume=True not resuming after the second epoch when breaking early#761

Merged
bhimrazy merged 2 commits into
Lightning-AI:mainfrom
philgzl:fix-parallel-streaming-dset-resume
Dec 8, 2025
Merged

Fix ParallelStreamingDataset with resume=True not resuming after the second epoch when breaking early#761
bhimrazy merged 2 commits into
Lightning-AI:mainfrom
philgzl:fix-parallel-streaming-dset-resume

Conversation

@philgzl
Copy link
Copy Markdown
Contributor

@philgzl philgzl commented Nov 26, 2025

Before submitting
  • Was this discussed/agreed via a Github issue? (no need for typos and docs improvements)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure to update the docs?
  • Did you write any new necessary tests?

What does this PR do?

Fixes #760.

Added related tests.

PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in GitHub issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃

@codecov
Copy link
Copy Markdown

codecov Bot commented Nov 26, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 80%. Comparing base (825586b) to head (2829273).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@         Coverage Diff         @@
##           main   #761   +/-   ##
===================================
  Coverage    80%    80%           
===================================
  Files        52     52           
  Lines      7347   7348    +1     
===================================
+ Hits       5890   5891    +1     
  Misses     1457   1457           
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Copy Markdown
Collaborator

@bhimrazy bhimrazy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@bhimrazy bhimrazy merged commit 4195db0 into Lightning-AI:main Dec 8, 2025
49 of 51 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ParallelStreamingDataset with resume=True does not resume after the second epoch when breaking early

4 participants