Summary
Pydantic AI's OpenTelemetry instrumentation supports InstrumentationSettings(include_content=False) to exclude message content — user prompts, model completions, tool call arguments and responses — from exported telemetry, so that agents can be monitored without sending sensitive content to the observability backend.
Retry prompts — the feedback messages Pydantic AI sends back to the model when its output fails validation or an output validator raises ModelRetry — were not covered by this setting when they were not associated with a tool call, as with structured output modes that don't use tool calls (e.g. NativeOutput or PromptedOutput) and output validators applied to text output. Their full content was recorded on model request spans even with include_content=False. Since validation feedback can quote invalid values from the model's response, message content that the setting was configured to withhold could reach the telemetry backend.
Details
Retry prompts without an associated tool call were serialized into span message attributes (such as gen_ai.input.messages and pydantic_ai.all_messages) as regular text parts, and this path did not apply the include_content check that every other message part applies. Retry prompts tied to a tool call, and all other message content, were redacted correctly.
Impact
This does not grant an attacker any new access to the agent or its data: the content is only visible to whoever can read the exported telemetry. The exposure matters when traces are exported to a destination whose audience is broader or less trusted than the agent's own data — a third-party observability vendor or a shared backend — which is the scenario include_content=False exists for. The disclosed content is limited to retry feedback: validation error details, which can quote invalid values from the model's output, or output-validator ModelRetry messages.
Who Is Affected
You are affected if you enabled instrumentation with include_content=False and your agent can produce retry prompts outside of tool calls: it uses a structured output mode that doesn't rely on tool calls (such as NativeOutput or PromptedOutput), or output validators on text output. You are not affected if you don't set include_content=False, or if all of your agent's retries come from tool calls (including the default tool-based structured output mode), which were redacted correctly.
Remediation
Upgrade to a patched version; retry prompt content now honors include_content=False like all other message content. If you rely on this setting, consider reviewing previously exported traces for retry prompt content that should not have been recorded.
Workaround for Unpatched Versions
If you cannot upgrade, scrub or drop the message attributes (gen_ai.input.messages, gen_ai.output.messages, pydantic_ai.all_messages) in your telemetry pipeline (for example with an OpenTelemetry Collector processor), or use tool-based structured output modes, whose retry feedback honors include_content=False.
Credits
Reported privately by the University of Sydney Security Research Team (Liyi Zhou, Ziyue Wang, Strick, Maurice, Chenchen Yu), and later independently discovered and reported publicly, with a fix, by @sean-kim05.
Summary
Pydantic AI's OpenTelemetry instrumentation supports
InstrumentationSettings(include_content=False)to exclude message content — user prompts, model completions, tool call arguments and responses — from exported telemetry, so that agents can be monitored without sending sensitive content to the observability backend.Retry prompts — the feedback messages Pydantic AI sends back to the model when its output fails validation or an output validator raises
ModelRetry— were not covered by this setting when they were not associated with a tool call, as with structured output modes that don't use tool calls (e.g.NativeOutputorPromptedOutput) and output validators applied to text output. Their full content was recorded on model request spans even withinclude_content=False. Since validation feedback can quote invalid values from the model's response, message content that the setting was configured to withhold could reach the telemetry backend.Details
Retry prompts without an associated tool call were serialized into span message attributes (such as
gen_ai.input.messagesandpydantic_ai.all_messages) as regular text parts, and this path did not apply theinclude_contentcheck that every other message part applies. Retry prompts tied to a tool call, and all other message content, were redacted correctly.Impact
This does not grant an attacker any new access to the agent or its data: the content is only visible to whoever can read the exported telemetry. The exposure matters when traces are exported to a destination whose audience is broader or less trusted than the agent's own data — a third-party observability vendor or a shared backend — which is the scenario
include_content=Falseexists for. The disclosed content is limited to retry feedback: validation error details, which can quote invalid values from the model's output, or output-validatorModelRetrymessages.Who Is Affected
You are affected if you enabled instrumentation with
include_content=Falseand your agent can produce retry prompts outside of tool calls: it uses a structured output mode that doesn't rely on tool calls (such asNativeOutputorPromptedOutput), or output validators on text output. You are not affected if you don't setinclude_content=False, or if all of your agent's retries come from tool calls (including the default tool-based structured output mode), which were redacted correctly.Remediation
Upgrade to a patched version; retry prompt content now honors
include_content=Falselike all other message content. If you rely on this setting, consider reviewing previously exported traces for retry prompt content that should not have been recorded.Workaround for Unpatched Versions
If you cannot upgrade, scrub or drop the message attributes (
gen_ai.input.messages,gen_ai.output.messages,pydantic_ai.all_messages) in your telemetry pipeline (for example with an OpenTelemetry Collector processor), or use tool-based structured output modes, whose retry feedback honorsinclude_content=False.Credits
Reported privately by the University of Sydney Security Research Team (Liyi Zhou, Ziyue Wang, Strick, Maurice, Chenchen Yu), and later independently discovered and reported publicly, with a fix, by @sean-kim05.