Skip to content

Conversation

@singh1203
Copy link
Contributor

Fixes: #71

Rationale for this change

Ensures PlainFixedLenByteArrayEncoder handles null values gracefully in non-nullable fields by writing zero-filled bytes instead of panicking, maintaining consistency with DictFixedLenByteArrayEncoder.

What changes are included in this PR?

  • Replaces panics on null values with writing zero-filled bytes to maintain byte alignment.
  • Ensures encoding consistency between dictionary and plain encoding.
  • Prevents unexpected failures when handling schema inconsistencies.

Are these changes tested?

Yes:

  • Passed existing test suite.
  • Verified with local Arrow-Go modifications.

Are there any user-facing changes?

Not Sure (Need possible guidance if required)

@singh1203 singh1203 changed the title fix: (GH-#71) Handle null values in PlainFixedLenByteArrayEncoder Fix: Handle null values in PlainFixedLenByteArrayEncoder gracefully Mar 20, 2025
@singh1203 singh1203 marked this pull request as ready for review March 20, 2025 12:57
@singh1203 singh1203 requested a review from zeroshade as a code owner March 20, 2025 12:57
Copy link
Member

@zeroshade zeroshade left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add a new test that would fail without this change?

@singh1203
Copy link
Contributor Author

Can we add a new test that would fail without this change?

Sure @zeroshade I'm on it.

@singh1203 singh1203 requested a review from zeroshade March 24, 2025 06:18
@zeroshade zeroshade merged commit 601311b into apache:main Mar 24, 2025
22 of 23 checks passed
@singh1203
Copy link
Contributor Author

Thank you. @zeroshade

@singh1203 singh1203 deleted the ByteArrayEncoder branch March 24, 2025 14:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Go][Parquet] PlainFixedLenByteArrayEncoder behaves differently from DictFixedLenByteArrayEncoder with null values where schema has Nullable: false

2 participants