Skip to content

refactor: reduce internal EagerSnapshot usage#4567

Draft
ethan-tyler wants to merge 6 commits into
delta-io:mainfrom
ethan-tyler:refactor/eagersnapshot-internal-cleanup
Draft

refactor: reduce internal EagerSnapshot usage#4567
ethan-tyler wants to merge 6 commits into
delta-io:mainfrom
ethan-tyler:refactor/eagersnapshot-internal-cleanup

Conversation

@ethan-tyler

Copy link
Copy Markdown
Collaborator

This removes internal EagerSnapshot dependencies where the code can use Snapshot, Arc<Snapshot>, active add APIs, tombstone APIs, DeltaScanNext, or FileSelection directly. Public compatibility stays intact.

Signed-off-by: Ethan Urbanski <ethan@urbanskitech.com>
Signed-off-by: Ethan Urbanski <ethan@urbanskitech.com>
Signed-off-by: Ethan Urbanski <ethan@urbanskitech.com>
@github-actions github-actions Bot added the binding/rust Issues for the Rust crate label Jun 25, 2026
@codecov

codecov Bot commented Jun 25, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 86.71096% with 40 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.03%. Comparing base (e87d969) to head (210d308).

Files with missing lines Patch % Lines
...re/src/delta_datafusion/table_provider/next/mod.rs 78.37% 0 Missing and 16 partials ⚠️
crates/core/src/delta_datafusion/find_files.rs 72.34% 0 Missing and 13 partials ⚠️
crates/core/src/operations/vacuum.rs 89.04% 0 Missing and 8 partials ⚠️
crates/core/src/operations/filesystem_check.rs 50.00% 0 Missing and 1 partial ⚠️
crates/core/src/operations/merge/mod.rs 75.00% 0 Missing and 1 partial ⚠️
crates/core/src/table/state.rs 0.00% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4567      +/-   ##
==========================================
+ Coverage   80.00%   80.03%   +0.02%     
==========================================
  Files         179      179              
  Lines       64140    64286     +146     
  Branches    64140    64286     +146     
==========================================
+ Hits        51315    51449     +134     
+ Misses       9834     9825       -9     
- Partials     2991     3012      +21     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@ethan-tyler ethan-tyler marked this pull request as ready for review June 25, 2026 10:35
@ethan-tyler ethan-tyler enabled auto-merge (rebase) June 25, 2026 10:41

let prepared_append = prepare_write(WritePreparationInput {
snapshot: Some(table_with_snapshot.snapshot().unwrap().snapshot()),
snapshot: Some(

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now with even more .snapshot() !!

Signed-off-by: Ethan Urbanski <ethan@urbanskitech.com>
@ethan-tyler ethan-tyler force-pushed the refactor/eagersnapshot-internal-cleanup branch from 7c5041f to 42bc8d1 Compare June 25, 2026 16:38

@hntd187 hntd187 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll be honest, I think this may shift the usage instead of actually reducing it. There are a lot of places where there was no .snapshot() and now there is a call and a lot of places where there are now more .snapshot() on top of the prior call sites.

I think this needs a bit more re-thinking in general and I think we should just rip the band-aid off. Why did we create different kinds of snapshots in the first place? Because the implementation details are different right? If that is the case is this not something we can collate into a trait and just pass a single "Snapshot" around instead of having to do this snapshot().snapshot().snapshot() things, which seems really ugly. If the different kinds of snapshots are so different they cannot be represented in a single trait then let's discuss more.

@ethan-tyler ethan-tyler marked this pull request as draft July 6, 2026 18:49
auto-merge was automatically disabled July 6, 2026 18:49

Pull request was converted to draft

@ethan-tyler

Copy link
Copy Markdown
Collaborator Author

going to keep this as a draft for now with it likely being superseded. A new issue was created to track a more robust snapshot design and future state here #4584.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

binding/rust Issues for the Rust crate

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

2 participants