Repository navigation
FIX: Keep "None", "null" and "N/A" text when loading a human-labeled CSV - #4481
devjoinedthechat (devjoinedthechat) wants to merge 1 commit into
Conversation
|
devjoinedthechat (@devjoinedthechat) please read the following Contributor License Agreement(CLA). If you agree with the CLA, please reply with the following information.
Contributor License AgreementContribution License AgreementThis Contribution License Agreement (“Agreement”) is agreed to by the party signing below (“You”),
|
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues remain for the change preserving literal text in human-labeled CSV datasets.
No open issues 🎉
What changed in this PR
Fixes #4480 by preserving literal text such as None, null, and N/A when loading human-labeled CSV datasets for scorer evaluation.
Changes:
- Treats only empty cells as missing in both UTF-8 and latin-1 CSV reads.
- Adds regression tests for missing-value-like text across objective and harm datasets, while checking that empty required cells remain rejected.
| File | Description |
|---|---|
tests/unit/score/test_human_labeled_dataset.py |
Tests text preservation across both encodings and dataset types, plus empty-cell rejection. |
pyrit/score/scorer_evaluation/human_labeled_dataset.py |
Updates and documents CSV missing-value handling for both encoding paths. |
Review effort: Balanced
Description
Closes #4480
HumanLabeledDataset.from_csvlet pandas treat cells such asNone,null,NA,N/Aandnanas missing, so a dataset where a response, objective or harm category is one of those words failed withColumn 'assistant_response' contains NaN values.keep_default_na=False, na_values=[""], so only an empty cell counts as missing.Before / after for the row
"None",Say nothing,0:ValueError: Column 'assistant_response' contains NaN values.None.