fix(core): preserve guardrail validation with Anthropic schemas (#15739)

## Summary

Fix guardrail processors with Anthropic structured output by applying
the existing Anthropic schema compatibility layer when building AI SDK
v5 response formats.

Anthropic rejects JSON Schema numeric bounds like `minimum` and
`maximum` on structured output schemas. The guardrail processors still
define scores with strict Zod validation (`z.number().min(0).max(1)`),
but the provider-facing JSON Schema now strips unsupported numeric
bounds for Anthropic before the request is sent.

This preserves the previous local validation semantics for generated
output while avoiding Anthropic request failures that caused guardrails
to fail open.

## What changed

- Pass model info into core response-format schema generation so
Anthropic compatibility can run on direct AI SDK v5 structured output.
- Keep guardrail score/confidence schemas strict with Zod
`min(0).max(1)`.
- Remove the temporary manual score validation/clamping path.
- Add regression tests for Anthropic-compatible provider schemas and
preserved local Zod validation.

## Test plan

- `pnpm --filter ./packages/core exec vitest run
src/processors/processors/prompt-injection-detector.test.ts
src/processors/processors/pii-detector.test.ts
src/processors/processors/moderation.test.ts
src/stream/base/schema.test.ts`
- `pnpm --filter ./packages/schema-compat test --
provider-compats/anthropic.test.ts`
- `pnpm --filter ./packages/core check`

Note: `pnpm test:core:zod` and `pnpm --filter ./packages/core
typecheck:zod-compat` were requested by package instructions, but those
scripts are not present in the current root/core `package.json` files.


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## ELI5

The PR fixes guardrail processors to work properly with Anthropic AI
models while keeping their safety checks intact. Anthropic doesn't
accept certain validation constraints in schemas (like numeric ranges),
so the code now removes those from what's sent to Anthropic—but keeps
the strict validation on the backend to ensure guardrails still work
correctly.

---

## Overview

This PR ensures that guardrail processors (moderation, PII detection,
prompt-injection detection) remain locally strict while producing
Anthropic-compatible structured-output schemas for AI SDK v5. The core
issue is that Anthropic rejects JSON Schema numeric bounds
(`minimum`/`maximum`), so the fix applies an existing Anthropic schema
compatibility layer when building provider-facing response formats.

## Changes

### Schema Compatibility Layer
- Updated `getTransformedSchema` and `getResponseFormat` in
`packages/core/src/stream/base/schema.ts` to accept an optional `model`
parameter (`SchemaModelInfo`)
- When model info is provided, applies `AnthropicSchemaCompatLayer` to
strip numeric bounds from JSON schemas sent to Anthropic
- Preserves existing behavior when no model info is provided
- Added new exported type `SchemaModelInfo` to support passing model
context

### Response Format Generation
- Modified `packages/core/src/stream/aisdk/v5/execute.ts` to pass model
context (`provider`, `modelId`, `supportsStructuredOutputs: true`) into
`getResponseFormat`
- Enables provider-specific schema transformations based on model
capabilities

### Guardrail Validation Tests
- Added new tests for moderation, PII detection, and prompt-injection
detection verifying Anthropic schema compatibility
- Tests confirm that generated schemas for Anthropic exclude numeric
bounds (`minimum`/`maximum`) while still containing score/confidence
fields
- Added runtime validation tests simulating out-of-range values to
verify fail-open behavior with warning logs
- Tests confirm local Zod validation remains strict with numeric
constraints (`z.number().min(0).max(1)`)

### Schema Compatibility Validation
- Added unit test for `AnthropicSchemaCompatLayer` in
`packages/schema-compat/src/provider-compats/anthropic.test.ts`
verifying numeric bounds are stripped while Zod validation behavior is
preserved

## Impact
- Guardrail processors maintain strict local validation semantics
- Schemas sent to Anthropic are compatible with Anthropic's structured
output requirements
- Removed temporary manual score validation/clamping path
- Regression tests ensure both Anthropic compatibility and preserved
local validation work correctly

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Mastra Code (openai/gpt-5.5) <noreply@mastra.ai>

Author: roaminro

.changeset/busy-doodles-begin.md

@@ -0,0 +1,5 @@
+---
+'@mastra/core': patch
+---
+
+Fixed guardrail processor schemas so structured output works with Anthropic models.

packages/core/src/processors/processors/moderation.test.ts

@@ -2,6 +2,7 @@ import { MockLanguageModelV1 } from '@internal/ai-sdk-v4/test';
 import { describe, it, expect, vi, beforeEach } from 'vitest';
 import type { MastraDBMessage } from '../../agent/message-list';
 import { TripWire } from '../../agent/trip-wire';
+import { MastraLanguageModelV2Mock } from '../../loop/test-utils/MastraLanguageModelV2Mock';
 import type { ChunkType } from '../../stream';
 import { ChunkFrom } from '../../stream/types';
 import type { ModerationResult } from './moderation';
@@ -700,4 +701,53 @@ describe('ModerationProcessor', () => {
       expect(mockAbort).not.toHaveBeenCalled();
     });
   });
+
+  describe('structured output schema compatibility', () => {
+    it('should not send number bounds to Anthropic in score schemas', async () => {
+      const mockResult = createMockModerationResult(false);
+      const mockModel = new MastraLanguageModelV2Mock({
+        provider: 'anthropic',
+        modelId: 'claude-3-5-sonnet',
+        doGenerate: async () => ({
+          rawCall: { rawPrompt: null, rawSettings: {} },
+          finishReason: 'stop',
+          usage: { inputTokens: 10, outputTokens: 20, totalTokens: 30 },
+          content: [{ type: 'text', text: JSON.stringify(mockResult) }],
+          warnings: [],
+        }),
+      });
+
+      const moderator = new ModerationProcessor({ model: mockModel });
+
+      await moderator.processInput({ messages: [createTestMessage('Safe content')], abort: vi.fn() as any });
+
+      const responseFormat = mockModel.doGenerateCalls[0].responseFormat;
+      expect(responseFormat?.type).toBe('json');
+      const schema = responseFormat?.type === 'json' ? responseFormat.schema : undefined;
+      const schemaJson = JSON.stringify(schema);
+      expect(schemaJson).toContain('score');
+      expect(schemaJson).not.toContain('minimum');
+      expect(schemaJson).not.toContain('maximum');
+    });
+
+    it('should reject scores outside the 0-1 range at runtime', async () => {
+      const model = setupMockModel({
+        object: {
+          category_scores: [{ category: 'hate', score: 1.2 }],
+          reason: 'Flagged content',
+        },
+      });
+      const moderator = new ModerationProcessor({ model, strategy: 'warn', includeScores: true });
+      const consoleSpy = vi.spyOn(console, 'warn').mockImplementation(() => {});
+
+      await moderator.processInput({ messages: [createTestMessage('Flagged content')], abort: vi.fn() as any });
+
+      expect(consoleSpy).toHaveBeenCalledWith(
+        '[ModerationProcessor] Agent moderation failed, allowing content:',
+        expect.any(Error),
+      );
+
+      consoleSpy.mockRestore();
+    });
+  });
 });

packages/core/src/processors/processors/pii-detector.test.ts

@@ -3,6 +3,7 @@ import { MockLanguageModelV1 } from '@internal/ai-sdk-v4/test';
 import { describe, it, expect, vi, beforeEach } from 'vitest';
 import type { MastraDBMessage } from '../../agent/message-list';
 import { TripWire } from '../../agent/trip-wire';
+import { MastraLanguageModelV2Mock } from '../../loop/test-utils/MastraLanguageModelV2Mock';
 import type { ChunkType } from '../../stream';
 import { ChunkFrom } from '../../stream/types';
 import type { PIIDetectionResult, PIIDetection } from './pii-detector';
@@ -1139,4 +1140,73 @@ describe('PIIDetector', () => {
       expect(result).toEqual(messages); // Should return original messages on failure
     });
   });
+
+  describe('structured output schema compatibility', () => {
+    it('should not send number bounds to Anthropic in score or confidence schemas', async () => {
+      const mockResult = createMockPIIResult();
+      const mockModel = new MastraLanguageModelV2Mock({
+        provider: 'anthropic',
+        modelId: 'claude-3-5-sonnet',
+        doGenerate: async () => ({
+          rawCall: { rawPrompt: null, rawSettings: {} },
+          finishReason: 'stop',
+          usage: { inputTokens: 10, outputTokens: 20, totalTokens: 30 },
+          content: [{ type: 'text', text: JSON.stringify(mockResult) }],
+          warnings: [],
+        }),
+      });
+
+      const detector = new PIIDetector({ model: mockModel });
+
+      await detector.processInput({ messages: [createTestMessage('Hello world')], abort: vi.fn() as any });
+
+      const responseFormat = mockModel.doGenerateCalls[0].responseFormat;
+      expect(responseFormat?.type).toBe('json');
+      const schema = responseFormat?.type === 'json' ? responseFormat.schema : undefined;
+      const schemaJson = JSON.stringify(schema);
+      expect(schemaJson).toContain('score');
+      expect(schemaJson).toContain('confidence');
+      expect(schemaJson).not.toContain('minimum');
+      expect(schemaJson).not.toContain('maximum');
+    });
+
+    it('should reject scores outside the 0-1 range at runtime', async () => {
+      const model = setupMockModel({
+        categories: [{ type: 'email', score: 1.2 }],
+        detections: null,
+      });
+      const detector = new PIIDetector({ model, strategy: 'warn', threshold: 1.1 });
+      const consoleSpy = vi.spyOn(console, 'warn').mockImplementation(() => {});
+
+      await detector.processInput({
+        messages: [createTestMessage('Hello world')],
+        abort: vi.fn() as any,
+      });
+
+      expect(consoleSpy).toHaveBeenCalledWith(
+        '[PIIDetector] Detection agent failed, allowing content:',
+        expect.any(Error),
+      );
+
+      consoleSpy.mockRestore();
+    });
+
+    it('should reject confidence values outside the 0-1 range at runtime', async () => {
+      const model = setupMockModel({
+        categories: null,
+        detections: [{ type: 'email', value: 'test@example.com', confidence: -0.2, start: 0, end: 16 }],
+      });
+      const detector = new PIIDetector({ model, strategy: 'warn', threshold: 0 });
+      const consoleSpy = vi.spyOn(console, 'warn').mockImplementation(() => {});
+
+      await detector.processInput({ messages: [createTestMessage('Hello world')], abort: vi.fn() as any });
+
+      expect(consoleSpy).toHaveBeenCalledWith(
+        '[PIIDetector] Detection agent failed, allowing content:',
+        expect.any(Error),
+      );
+
+      consoleSpy.mockRestore();
+    });
+  });
 });

packages/core/src/processors/processors/prompt-injection-detector.test.ts

@@ -723,4 +723,51 @@ describe('PromptInjectionDetector', () => {
       });
     });
   });
+
+  describe('structured output schema compatibility', () => {
+    it('should not send number bounds to Anthropic in score schemas', async () => {
+      const mockResult = createMockDetectionResult(false);
+      const mockModel = new MastraLanguageModelV2Mock({
+        provider: 'anthropic',
+        modelId: 'claude-3-5-sonnet',
+        doGenerate: async () => ({
+          rawCall: { rawPrompt: null, rawSettings: {} },
+          finishReason: 'stop',
+          usage: { inputTokens: 10, outputTokens: 20, totalTokens: 30 },
+          content: [{ type: 'text', text: JSON.stringify(mockResult) }],
+          warnings: [],
+        }),
+      });
+
+      const detector = new PromptInjectionDetector({ model: mockModel });
+
+      await detector.processInput({ messages: [createTestMessage('Test message', 'user')], abort: vi.fn() as any });
+
+      const responseFormat = mockModel.doGenerateCalls[0].responseFormat;
+      expect(responseFormat?.type).toBe('json');
+      const schema = responseFormat?.type === 'json' ? responseFormat.schema : undefined;
+      const schemaJson = JSON.stringify(schema);
+      expect(schemaJson).toContain('score');
+      expect(schemaJson).not.toContain('minimum');
+      expect(schemaJson).not.toContain('maximum');
+    });
+
+    it('should fail open and warn when scores are outside the 0-1 range', async () => {
+      const model = setupMockModel({
+        categories: [{ type: 'injection', score: 1.2 }],
+        reason: 'Attack detected',
+      });
+      const detector = new PromptInjectionDetector({ model, strategy: 'warn', includeScores: true });
+      const consoleSpy = vi.spyOn(console, 'warn').mockImplementation(() => {});
+
+      await detector.processInput({ messages: [createTestMessage('Test message', 'user')], abort: vi.fn() as any });
+
+      expect(consoleSpy).toHaveBeenCalledWith(
+        '[PromptInjectionDetector] Detection agent failed, allowing content:',
+        expect.any(Error),
+      );
+
+      consoleSpy.mockRestore();
+    });
+  });
 });

packages/core/src/stream/aisdk/v5/execute.ts

@@ -87,7 +87,15 @@ export function execute<OUTPUT = undefined>({
       : 'direct'
     : undefined;
 
-  const responseFormat = structuredOutput?.schema ? getResponseFormat(structuredOutput?.schema) : undefined;
+  const responseFormat = structuredOutput?.schema
+    ? getResponseFormat(structuredOutput?.schema, {
+        model: {
+          provider: model.provider,
+          modelId: model.modelId,
+          supportsStructuredOutputs: true,
+        },
+      })
+    : undefined;
 
   let prompt = inputMessages;
 

packages/core/src/stream/base/schema.test.ts

@@ -18,7 +18,35 @@
 
 import { describe, it, expect } from 'vitest';
 import { z } from 'zod/v4';
-import { asJsonSchema } from './schema';
+import { asJsonSchema, getResponseFormat } from './schema';
+
+describe('getResponseFormat', () => {
+  it('applies Anthropic schema compatibility without removing local validation', async () => {
+    const schema = z.object({
+      score: z.number().min(0).max(1),
+    });
+
+    const responseFormat = getResponseFormat(schema, {
+      model: {
+        provider: 'anthropic',
+        modelId: 'claude-3-5-sonnet',
+        supportsStructuredOutputs: true,
+      },
+    });
+
+    expect(responseFormat.type).toBe('json');
+    const schemaJson = JSON.stringify(responseFormat.type === 'json' ? responseFormat.schema : undefined);
+    expect(schemaJson).toContain('score');
+    expect(schemaJson).not.toContain('minimum');
+    expect(schemaJson).not.toContain('maximum');
+
+    const validResult = await schema['~standard'].validate({ score: 0.5 });
+    expect(validResult).toEqual({ value: { score: 0.5 } });
+
+    const invalidResult = await schema['~standard'].validate({ score: 1.2 });
+    expect('issues' in invalidResult).toBe(true);
+  });
+});
 
 describe('asJsonSchema - Zod v4 transform compatibility', () => {
   describe('should handle schemas with transforms', () => {

packages/core/src/stream/base/schema.ts

@@ -1,7 +1,8 @@
 import type { JSONSchema7, Schema } from '@internal/ai-sdk-v5';
+import { AnthropicSchemaCompatLayer, applyCompatLayer } from '@mastra/schema-compat';
 import type { z as z3 } from 'zod/v3';
 import type { z as z4 } from 'zod/v4';
-import type { StandardSchemaWithJSON } from '../../schema';
+import type { PublicSchema, StandardSchemaWithJSON } from '../../schema';
 import { isStandardSchemaWithJSON, standardSchemaToJSONSchema } from '../../schema';
 
 export type PartialSchemaOutput<OUTPUT = undefined> = OUTPUT extends undefined ? undefined : Partial<OUTPUT>;
@@ -69,8 +70,27 @@ export function asJsonSchema(schema: StandardSchemaWithJSON | undefined): JSONSc
   return schema;
 }
 
-export function getTransformedSchema<OUTPUT = undefined>(schema?: StandardSchemaWithJSON<OUTPUT>) {
-  const jsonSchema = asJsonSchema(schema);
+type SchemaModelInfo = {
+  provider: string;
+  modelId: string;
+  supportsStructuredOutputs: boolean;
+};
+
+export function getTransformedSchema<OUTPUT = undefined>(
+  schema?: StandardSchemaWithJSON<OUTPUT>,
+  options?: { model?: SchemaModelInfo },
+) {
+  if (!schema) {
+    return undefined;
+  }
+
+  const jsonSchema = options?.model
+    ? (applyCompatLayer({
+        schema: schema as PublicSchema<OUTPUT>,
+        compatLayers: [new AnthropicSchemaCompatLayer(options.model)],
+        mode: 'jsonSchema',
+      }) as JSONSchema7)
+    : asJsonSchema(schema);
 
   if (!jsonSchema) {
     return undefined;
@@ -119,7 +139,10 @@ export function getTransformedSchema<OUTPUT = undefined>(schema?: StandardSchema
   };
 }
 
-export function getResponseFormat(schema?: StandardSchemaWithJSON):
+export function getResponseFormat(
+  schema?: StandardSchemaWithJSON,
+  options?: { model?: SchemaModelInfo },
+):
   | {
       type: 'text';
     }
@@ -131,7 +154,7 @@ export function getResponseFormat(schema?: StandardSchemaWithJSON):
       schema?: JSONSchema7;
     } {
   if (schema) {
-    const transformedSchema = getTransformedSchema(schema);
+    const transformedSchema = getTransformedSchema(schema, options);
     return {
       type: 'json',
       schema: transformedSchema?.jsonSchema,

packages/schema-compat/src/provider-compats/anthropic.test.ts

@@ -1,4 +1,6 @@
 import { describe, it, expect } from 'vitest';
+import { z } from 'zod/v4';
+import { standardSchemaToJSONSchema } from '../standard-schema/standard-schema';
 import type { ModelInformation } from '../types';
 import { AnthropicSchemaCompatLayer } from './anthropic';
 import { createSuite } from './test-suite';
@@ -60,4 +62,26 @@ describe('AnthropicSchemaCompatLayer', () => {
       expect(layer.getSchemaTarget()).toBe('jsonSchema7');
     });
   });
+
+  describe('number bounds', () => {
+    it('should strip number bounds from JSON Schema while preserving Zod validation', async () => {
+      const schema = z.object({
+        score: z.number().min(0).max(1),
+      });
+      const layer = new AnthropicSchemaCompatLayer(modelInfo);
+      const compatSchema = layer.processToCompatSchema(schema);
+      const jsonSchema = standardSchemaToJSONSchema(compatSchema);
+      const schemaJson = JSON.stringify(jsonSchema);
+
+      expect(schemaJson).toContain('score');
+      expect(schemaJson).not.toContain('minimum');
+      expect(schemaJson).not.toContain('maximum');
+
+      const validResult = await compatSchema['~standard'].validate({ score: 0.5 });
+      expect(validResult).toEqual({ value: { score: 0.5 } });
+
+      const invalidResult = await compatSchema['~standard'].validate({ score: 1.2 });
+      expect('issues' in invalidResult).toBe(true);
+    });
+  });
 });