-
Notifications
You must be signed in to change notification settings - Fork 85
Fix panic when WriteDataPage fails #366
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
| for i, p := range w.pages { | ||
| defer p.Release() | ||
| if err = w.WriteDataPage(p); err != nil { | ||
| // To keep pages in consistent state, | ||
| // remove the pages that will be released using above defer call. | ||
| w.pages = w.pages[i+1:] | ||
| return err | ||
| } | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there an easy way to add a test case for this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added a unit test and updated PR description.
zeroshade
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Awesome, thanks! Will merge after CI finishes
|
@zeroshade Thanks for quick review! I am wondering if this can be back ported to latest stable release |
|
I actually think it's about time for an 18.3.0 release :) so I'll start preparing that |
|
Hi @zeroshade |
|
Hi! Sorry, I'll kick off the RC and the vote on the mailing list today. With luck I should be able to release v18.3.0 by Friday |
Rationale for this change
We saw crash with call stack:
On code inspection, i see that if
WriteDataPagefails,w.pageswill still be holding releasedBuffer/Pages.On error handling, we close the parquet file, which tries to access
w.pagesthroughTotalBytesWritten()- causing crash.What changes are included in this PR?
The PR includes a bug fix which removes the to-be released pages from
w.pages.Are these changes tested?
Added unit test.
Without the fix: test panics
With the fix:
Are there any user-facing changes?
No