Skip to content

FIX: Rebuild decision and Prompt Shield targets for capability detection - #4492

Open
Richard Lundeen (richlundeen) wants to merge 1 commit into
microsoft:mainfrom
richlundeen:richlundeen-rebuild-decision-promptshield
Open

Richard Lundeen (richlundeen) wants to merge 1 commit into
microsoft:mainfrom
richlundeen:richlundeen-rebuild-decision-promptshield

Conversation

@richlundeen

Copy link
Copy Markdown
Contributor

Description

The GUI's capability detection probes a private rebuilt copy of a target, and LiteLLMDecisionTarget (microsoft_decision) and PromptShieldTarget (azure_content_safety) failed at the rebuild step with "does not support reconstruction". This makes both rebuildable and gives detection a meaningful check for them.

  • LiteLLMDecisionTarget and PromptShieldTarget implement get_reconstruction_parameters, reusing the source's resolved key or token provider (identity mode reuses the minted provider instead of minting a new one). Credentials stay out of identifiers, and neither target holds a client, so the copy shares no connection state. Their endpoint, credential, and auth settings (plus the decision model name, whose provider prefix picks where LiteLLM sends the key) are copy-locked, as for OpenAI targets.
  • Discovery sends a target's capability_probe_text (default "test") for text probes. The decision target overrides it with a valid state/questions request and is no longer treated as a chat adapter, so detection runs one control request and keeps its declared capabilities. Prompt Shield already passes the plain-text control request and keeps its declarations.
  • Detection now rejects a target, or a round-robin child, that can't be rebuilt before any probe runs, with "Capability detection is not supported for because it can't be rebuilt."
  • Request-pacing sharing for rebuilt copies moved from OpenAITarget to PromptTarget.attach_reconstruction_source, so the new copies also count against their source's rate limit.

Related: #4486 also edits target_service.py and discover_target_capabilities.py, in different hunks.

LiteLLMDecisionTarget and PromptShieldTarget now supply reconstruction
parameters, reusing the source's resolved key or token provider, with
connection settings copy-locked. Discovery sends a target-provided probe
text so the decision target's control request is a valid decision request,
and the decision target no longer runs chat probes. Detection rejects
targets that can't be rebuilt with a clear message before probing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings November 21, 2026 05:31

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues remain for the target capability-detection changes.

No open issues 🎉

What changed in this PR

Enables GUI capability detection for LiteLLMDecisionTarget and PromptShieldTarget by making them safely rebuildable as private probe copies.

Changes:

  • Adds reconstruction that preserves resolved credentials and locks connection settings.
  • Uses target-specific probe text and preserves declared capabilities for non-chat adapters.
  • Rejects unrebuildable targets before probing and shares request pacing through the base target class.
File Description
tests/​unit/​prompt_target/​target/​test_prompt_shield_target.py Tests reconstruction, credential reuse, and connection locks.
tests/​unit/​prompt_target/​target/​test_litellm_decision_target.py Tests reconstructed requests and connection locks.
tests/​unit/​backend/​test_target_detection.py Covers detection for both targets and unsupported-target errors.
pyrit/​prompt_target/​prompt_shield_target.py Adds credential-preserving reconstruction and connection locks.
pyrit/​prompt_target/​openai/​openai_target.py Removes pacing-sharing logic now handled by the base class.
pyrit/​prompt_target/​litellm_decision_target.py Adds reconstruction, connection locks, and a valid probe payload.
pyrit/​prompt_target/​common/​prompt_target.py Defines default probe text and shares reconstruction pacing.
pyrit/​prompt_target/​common/​discover_target_capabilities.py Uses custom probe text and excludes decision targets from chat probes.
pyrit/​backend/​services/​target_service.py Checks reconstruction support before detection begins.

Review effort: Balanced

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants