Repository navigation
Open
Labels
datasetsPulling in external datasets into PyRITPulling in external datasets into PyRIThelp wantedExtra attention is neededExtra attention is needed
Description
Add a seed dataset loader for CoSafe, 1,400 multi-turn conversations in which the final harmful question refers back to earlier turns instead of naming the harmful thing directly.
Why it fits
A safety filter that looks at one message at a time can miss a request like "how do I make more of it?" when "it" was introduced two turns earlier. CoSafe tests exactly that: each conversation builds context over several turns and ends with a question that is only harmful once the reference is resolved.
Dataset facts
| Source | GitHub repository ErxinYu/CoSafe-Dataset |
| Licence | None stated in the repository. The loader fetches the files at run time, like every remote loader; no data is added to PyRIT. |
| Access | Open |
CoSafe datasets/ |
14 files, one per harm category (for example self_harm.json, privacy_violation.json), 100 conversations each |
Single Prompt/ |
14 files with the single-turn version of the same questions |
| Format | Despite the .json extension, the multi-turn files are JSON Lines: each line is one conversation, a list of {"role": ..., "content": ...} messages. In the file I sampled, conversations alternate user and assistant and end on a user turn. |
| Paper | Yu et al.: "CoSafe: Evaluating Large Language Model Safety in Multi-Turn Dialogue Coreference", EMNLP 2024, arXiv:2406.17626 |
Proposed shape
- A
CoSafeCategoryenum for the 14 categories, and a constructor argumentcategories: list[CoSafeCategory] | None = NonewhereNonemeans all categories. Several file names contain commas; map each enum member to its exact file name. - One seed group per conversation: one
SeedPromptper message, with itsroleand an increasingsequence, all sharing aprompt_group_id.mask_dataset.pyis an existing loader that builds multi-turn groups with roles this way. harm_categoriescomes from the file the conversation was read from.- Fetch with
_fetch_from_url. Check whether its JSON handling copes with JSON Lines content under a.jsonname, and handle that in the loader if it does not. - The
Single Prompt/files are out of scope unless you can show how they line up with the multi-turn conversations; if they do, the single-turn question belongs inmetadata.
Implementation notes
- Follow
.github/instructions/datasets.instructions.md. It covers everything a loader needs: subclassing_RemoteDatasetLoader, choosing betweenSeedPromptandSeedObjective, filter enums, per-seed metadata, class-level metadata, registration inremote/__init__.py, the BibTeX entry, the datasets doc page, and unit tests with mocked data. aya_redteaming_dataset.pyandtests/unit/datasets/test_aya_redteaming_dataset.pyare a good template for both the loader and its tests.- The schema above was read from the dataset card. Run your loader once against the real dataset before opening the PR and confirm column names, category values and row counts; say in the PR description that you did.
- Unit tests must mock the download. No test may fetch from the network.
Done when
- The loader exists, is registered, and its dataset name appears in
SeedDatasetProvider.get_all_dataset_names_async(). - Unit tests cover the happy path, every filter value, the empty-after-filter
ValueError, and invalid filter values. - The paper is cited in
doc/references.bibanddoc/bibliography.md, and the dataset is listed indoc/code/datasets/1_loading_datasets. - Unit tests and pre-commit pass.
How to contribute
- Claim it first. Before you start, comment on this issue to say you are taking it.
- First comment wins. If yours is the first claim comment, go ahead and open a PR; you do not need to wait for a reply. If someone has already commented, this issue is taken, so please pick a different one.
- Link this issue in your PR description.
Metadata
Metadata
Assignees
Labels
datasetsPulling in external datasets into PyRITPulling in external datasets into PyRIThelp wantedExtra attention is neededExtra attention is needed
Activity
Shizoqua commentedon Nov 21, 2026
I'd like to work on this. Could you assign it to me?