Skip to content

Backport #36443 to 2.0: fix CSV import crash on malformed quoted fields - #37214

Merged
yan-3005 merged 1 commit into
2.0from
fix/csv-unchecked-io-2.0
Nov 6, 2026
Merged

yan-3005 merged 1 commit into
2.0from
fix/csv-unchecked-io-2.0

Conversation

@yan-3005

@yan-3005 yan-3005 commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

Describe your changes:

Backport of #36443 to 2.0.

On 2.0, importing a CSV with a malformed quoted field (a stray character after a closing quote, or a quote that is never closed) fails with a server error: Commons CSV 1.12 (same version as main) wraps the lexer error in UncheckedIOException, which both EntityCsv.parse overloads let escape. Sync imports return HTTP 500 and importAsync jobs end FAILED with the raw CSVException text.

This backport:

  • catches UncheckedIOException in both parse overloads and reports it as PARSER_FAILURE, so the import returns ABORTED;
  • returns null from parse(csv, true) on a parser failure, so recursive database / schema / service / drive-service imports hit the existing records == null guard in importCsv(records, ...) instead of records.get(0) on an empty list;
  • words the abort reason as the line and the fix (CsvParseErrorMessage), e.g. line 2 has characters after the closing quote of a quoted value. ... and line 3 starts a quoted value that is never closed. ..., and fixes the "CSV filed" typo.

Conflict resolved: 2.0 does not have restoreFormulas (#36070 is not on 2.0), so only the new documentParserFailure helper was taken from that hunk. On 2.0, parse(csv) still uses .stream().toList(), which raises the same UncheckedIOException.

Blast radius: only malformed CSVs behave differently. The four 2.0 callers of parse(csv, recursive) all pass the result to importCsv(records, ...), which already returns early on null.

Type of change:

  • Bug fix

Tests:

Unit tests

  • Cherry-picked with the fix.
  • On 2.0 + this backport: EntityCsvTest (68), CsvParseErrorMessageTest (3), CsvParseTest (3), CsvUtilTest (10): 84 run, 0 failures.
  • Same tests against the original 2.0 EntityCsv.java: test_importMalformedCsvAbortsWithParserFailure and test_recursiveImportOfMalformedCsvAbortsWithParserFailure error with the escaped exception, so the bug is present on 2.0.

Live server evidence (from #36443 on main; same code path and Commons CSV version)

Before: HTTP 500 and a FAILED job

After: the import aborts and names the line and the fix

UI screen recording:

Not applicable — backend-only change.

Catch the UncheckedIOException Commons CSV raises for malformed quoted fields in both EntityCsv.parse overloads and report it as PARSER_FAILURE, return null from the recursive overload so recursive imports abort cleanly instead of failing with a 500, and word the abort reason as the line and the fix (characters after a closing quote, unclosed quote) instead of the raw lexer message.

(cherry picked from commit d44ad6e)
@yan-3005
yan-3005 requested a review from a team as a code owner November 6, 2026 04:43
@yan-3005 yan-3005 added safe to test Add this label to run secure Github workflows on PRs backend labels Nov 6, 2026
@yan-3005
yan-3005 requested review from TeddyCr and removed request for a team November 6, 2026 04:43
@github-actions

github-actions Bot commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

❌ PR checklist incomplete

This PR cannot be merged until the following are addressed on its linked issue:

  • No GitHub issue is linked. Link an issue in the Development section of the PR (or add Fixes #12345 to the description). For a same-org cross-repo issue, add Fixes open-metadata/<repo>#123 to the description.

The fields live on the linked issue in the Shipping project (open the issue → right sidebar → Projects). After you set them, re-run this check (or push a commit) — issue/project changes do not re-trigger it automatically.

Maintainers can bypass this check by adding the skip-pr-checks label.

@gitar-bot

gitar-bot Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Code Review ✅ Approved

🟡 Medium risk · Catching unchecked CSV lexer errors routes malformed imports to ABORTED instead of server errors, so those imports may be rejected without importing records.

Backports the CSV import fix so malformed quoted fields abort with a clear parser failure instead of an HTTP 500 or a FAILED async job. Both parse overloads now catch UncheckedIOException and return PARSER_FAILURE, and recursive imports exit cleanly on a null result. No issues found.

Verification

Rules No rules evaluated

Functional validation Not enabled

Set up

Auto-approval Not enabled

Set up

For more verification details, comment gitar display:verbose.

Options

Display: compact → Showing compact verification results.

Comment with these commands to change the behavior for this request:

Compact
gitar display:verbose         

Was this helpful? React with 👍 / 👎 | Powered by Gitar — free for open source

This branch was previously deployed

1 inactive deployment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend safe to test Add this label to run secure Github workflows on PRs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants