Skip to content

Keep CSV quoting in LLMFileAnalysisOperator sample rows - #76389

Merged
kaxil merged 2 commits into
apache:mainfrom
aarmens702-hub:keep-csv-quoting-in-file-analysis
Nov 8, 2026
Merged

kaxil merged 2 commits into
apache:mainfrom
aarmens702-hub:keep-csv-quoting-in-file-analysis

Conversation

@aarmens702-hub

Copy link
Copy Markdown
Contributor

LLMFileAnalysisOperator showed CSV sample rows with their quotes removed, so a value holding a comma or a line break looked like several values to the model. Values that need quoting are now quoted the way CSV quotes them, and all other values render as before.

Tested with a new test_csv_values_keep_their_quotes in test_file_analysis.py (a quoted comma, an escaped quote and a line break). It fails on main and passes with this change. test_file_analysis.py, test_llm_file_analysis.py and test_object_storage.py pass (120 passed, 8 skipped because fastavro isn't installed). ruff format, ruff check and mypy on the changed file are clean. I didn't run the full prek set or breeze.

closes: #76386


Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

Generated-by: Claude Code (Opus 5.5) following the guidelines

The single value with both a quote and a line break meant dropping either
character from the quoting set still passed. One case per character, plus a
plain value that must stay unquoted.
@kaxil
kaxil enabled auto-merge (squash) November 4, 2026 12:37
@kaxil
kaxil merged commit 84a2037 into apache:main Nov 8, 2026
79 checks passed
@boring-cyborg

boring-cyborg Bot commented Nov 8, 2026

Copy link
Copy Markdown

Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

LLMFileAnalysisOperator drops CSV quoting, so values with commas look like extra columns

2 participants