Skip to content

BUG: remove_seeds_from_memory / remove_seed_groups_from_memory delete the whole dataset when given an empty id or hash list #4473

Open

Description

Describe the bug

remove_seeds_from_memory_async(dataset_name="ds", prompt_group_ids=[]) deletes every seed in ds. remove_seed_groups_from_memory_async(dataset_name="ds", value_sha256=[]) does the same.

Root cause:

  • _build_seed_filter_conditions (pyrit/memory/memory_interface.py, the if value_sha256: check at line 5077 and if prompt_group_ids: at line 5083) uses a truthiness check, so an empty list adds no condition.
  • The remaining dataset_name filter satisfies the "at least one filter" guard, and the delete matches every row in the dataset.

Elsewhere in MemoryInterface, an explicitly empty id list matches nothing:

  • get_message_pieces_async(prompt_ids=[]) returns [].
  • get_scores_async(score_ids=[]) returns [].

The empty list is a natural input here, e.g. a UI or script where the user deselected every group before clicking "remove". For a read, "empty means no filter" might be arguable; for a delete it removes data the caller didn't select.

Steps/Code to Reproduce

import asyncio
import uuid

from pyrit.memory import CentralMemory
from pyrit.models import SeedPrompt
from pyrit.setup import IN_MEMORY, initialize_pyrit_async


async def add_three_seeds(memory, dataset_name):
    seeds = [
        SeedPrompt(value=f"{dataset_name}-{i}", data_type="text", dataset_name=dataset_name, prompt_group_id=uuid.uuid4())
        for i in range(3)
    ]
    await memory.add_seeds_to_memory_async(seeds=seeds, added_by="repro")


async def main():
    await initialize_pyrit_async(memory_db_type=IN_MEMORY, silent=True)
    memory = CentralMemory.get_memory_instance()

    # Elsewhere in MemoryInterface an explicitly empty id list matches nothing:
    print("get_message_pieces(prompt_ids=[]) ->", len(await memory.get_message_pieces_async(prompt_ids=[])))
    print("get_scores(score_ids=[])          ->", len(await memory.get_scores_async(score_ids=[])))

    await add_three_seeds(memory, "ds1")
    selected_group_ids: list[uuid.UUID] = []  # e.g. the user deselected every group in a picker
    removed = await memory.remove_seeds_from_memory_async(dataset_name="ds1", prompt_group_ids=selected_group_ids)
    print("remove_seeds_from_memory(dataset_name='ds1', prompt_group_ids=[]) removed", removed)
    print("seeds left in ds1:", len(await memory.get_seeds_async(dataset_name="ds1")))

    await add_three_seeds(memory, "ds2")
    removed = await memory.remove_seed_groups_from_memory_async(dataset_name="ds2", value_sha256=[])
    print("remove_seed_groups_from_memory(dataset_name='ds2', value_sha256=[]) removed", removed)
    print("seeds left in ds2:", len(await memory.get_seeds_async(dataset_name="ds2")))


asyncio.run(main())

Expected Results

An empty prompt_group_ids / value_sha256 list matches nothing, so 0 seeds are removed and the datasets keep all 3 seeds, consistent with prompt_ids=[] and score_ids=[].

Actual Results

get_message_pieces(prompt_ids=[]) -> 0
get_scores(score_ids=[])          -> 0
remove_seeds_from_memory(dataset_name='ds1', prompt_group_ids=[]) removed 3
seeds left in ds1: 0
remove_seed_groups_from_memory(dataset_name='ds2', value_sha256=[]) removed 3
seeds left in ds2: 0

Possible fix

Distinguish None from an empty list in _build_seed_filter_conditions (is not None), and make an empty list match nothing: either return 0 early in the remove paths, or add a false() condition. get_seeds with an empty list would then also return nothing, consistent with the other getters. I'm happy to open a PR with regression tests if you agree with that behaviour.

Versions

  • OS: macOS 27 (arm64)
  • Python version: 3.13.11
  • PyRIT version: 1.3.0.dev0, installed from main (e24801a7d) in editable mode with uv sync --locked

AI assistance: an AI coding assistant (Claude Code) helped me find these code paths, write the reproduction and draft this report. I ran the reproduction above against main @ e24801a7d; the output shown is from that run.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions