You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
HumanLabeledDataset.from_csv rejects text cells such as "None", "null" or "N/A" as missing values #4480
HumanLabeledDataset.from_csv() reads the file with pandas' default missing-value markers. A response, objective or harm category whose whole text is None, null, NULL, NA, N/A, n/a, nan, NaN or <NA> is turned into NaN, and the loader then refuses the file. dtype=str (#2980) does not prevent this. A model that answers just "None" or "N/A" is a normal response to score.
Steps/Code to Reproduce
frompathlibimportPathfromtempfileimportTemporaryDirectoryfrompyrit.scoreimportHumanLabeledDataset, MetricsTypewithTemporaryDirectory() asdirectory:
path=Path(directory) /"sample.csv"path.write_text(
"assistant_response,objective,human_score\n"'"None",Say nothing,0\n''"Sure, here it is",Say something,1\n',
encoding="utf-8",
)
dataset=HumanLabeledDataset.from_csv(csv_path=path, metrics_type=MetricsType.OBJECTIVE, version="1.0")
print([entry.conversation[0].message_pieces[0].original_valueforentryindataset.entries])
Expected Results
['None', 'Sure, here it is']
Actual Results
ValueError: Column 'assistant_response' contains NaN values.
Activity