Repository navigation
FIX: Rebuild decision and Prompt Shield targets for capability detection - #4492
Open
Richard Lundeen (richlundeen) wants to merge 1 commit into
Open
Richard Lundeen (richlundeen) wants to merge 1 commit into
Richard Lundeen (richlundeen) wants to merge 1 commit into
Conversation
LiteLLMDecisionTarget and PromptShieldTarget now supply reconstruction parameters, reusing the source's resolved key or token provider, with connection settings copy-locked. Discovery sends a target-provided probe text so the decision target's control request is a valid decision request, and the decision target no longer runs chat probes. Detection rejects targets that can't be rebuilt with a clear message before probing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues remain for the target capability-detection changes.
No open issues 🎉
What changed in this PR
Enables GUI capability detection for LiteLLMDecisionTarget and PromptShieldTarget by making them safely rebuildable as private probe copies.
Changes:
- Adds reconstruction that preserves resolved credentials and locks connection settings.
- Uses target-specific probe text and preserves declared capabilities for non-chat adapters.
- Rejects unrebuildable targets before probing and shares request pacing through the base target class.
| File | Description |
|---|---|
tests/unit/prompt_target/target/test_prompt_shield_target.py |
Tests reconstruction, credential reuse, and connection locks. |
tests/unit/prompt_target/target/test_litellm_decision_target.py |
Tests reconstructed requests and connection locks. |
tests/unit/backend/test_target_detection.py |
Covers detection for both targets and unsupported-target errors. |
pyrit/prompt_target/prompt_shield_target.py |
Adds credential-preserving reconstruction and connection locks. |
pyrit/prompt_target/openai/openai_target.py |
Removes pacing-sharing logic now handled by the base class. |
pyrit/prompt_target/litellm_decision_target.py |
Adds reconstruction, connection locks, and a valid probe payload. |
pyrit/prompt_target/common/prompt_target.py |
Defines default probe text and shares reconstruction pacing. |
pyrit/prompt_target/common/discover_target_capabilities.py |
Uses custom probe text and excludes decision targets from chat probes. |
pyrit/backend/services/target_service.py |
Checks reconstruction support before detection begins. |
Review effort: Balanced
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
The GUI's capability detection probes a private rebuilt copy of a target, and
LiteLLMDecisionTarget(microsoft_decision) andPromptShieldTarget(azure_content_safety) failed at the rebuild step with "does not support reconstruction". This makes both rebuildable and gives detection a meaningful check for them.LiteLLMDecisionTargetandPromptShieldTargetimplementget_reconstruction_parameters, reusing the source's resolved key or token provider (identity mode reuses the minted provider instead of minting a new one). Credentials stay out of identifiers, and neither target holds a client, so the copy shares no connection state. Their endpoint, credential, and auth settings (plus the decision model name, whose provider prefix picks where LiteLLM sends the key) are copy-locked, as for OpenAI targets.capability_probe_text(default"test") for text probes. The decision target overrides it with a valid state/questions request and is no longer treated as a chat adapter, so detection runs one control request and keeps its declared capabilities. Prompt Shield already passes the plain-text control request and keeps its declarations.OpenAITargettoPromptTarget.attach_reconstruction_source, so the new copies also count against their source's rate limit.Related: #4486 also edits
target_service.pyanddiscover_target_capabilities.py, in different hunks.