feat(memory): support inline json prompt injection#18652
Merged
Conversation
🦋 Changeset detectedLatest commit: 26a8be6 The changes in this PR will be included in the next version bump. This PR includes changesets to release 22 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (3)
✅ Files skipped from review due to trivial changes (1)
🚧 Files skipped from review as they are similar to previous changes (2)
WalkthroughExtends ChangesInline JSON Prompt Injection
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Suggested reviewers
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
|
The latest updates on your projects. Learn more about Vercel for GitHub.
1 Skipped Deployment
|
Contributor
PR triageLinked issue check skipped for core contributor @TylerBarnes. PR complexity score
Applied label: Changed test gateChanged tests failed against the base branch as expected. Label: |
dane-ai-mastra
Bot
added
the
tests: green ✅
abhiaiyer91
approved these changes
Jun 30, 2026
Widens the jsonPromptInjection type from boolean to boolean | 'system' | 'inline'. The 'inline' mode injects JSON schema instructions into the latest user message instead of the system prompt, preserving prompt cache for providers with prefix-based caching. Adds feature flag 'json-prompt-injection:inline' for runtime capability detection. Co-Authored-By: Mastra Code (crof/glm-5.2) <noreply@mastra.ai>
Co-Authored-By: Mastra Code (crof/glm-5.2) <noreply@mastra.ai>
TylerBarnes
force-pushed
the
feat/inline-json-injection
branch
from
0ede67b to
927182f
Compare
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.changeset/quick-dingos-eat.md:
- Around line 5-13: This changeset reads too much like internal implementation
detail and is missing a user-facing API example. Update the entry to highlight
the developer outcome of using structuredOutput.jsonPromptInjection: 'inline'
and add a short public example showing agent.generate with
structuredOutput.schema and jsonPromptInjection set to 'inline'. Keep the
wording focused on what changes for users rather than the runtime feature flag
or type widening, so the changeset clearly communicates the new API behavior.
In `@packages/core/src/agent/utils.test.ts`:
- Around line 59-72: Add a retry-preservation test for the explicit 'system'
jsonPromptInjection mode in tryGenerateWithJsonFallback(), since the current
case only verifies 'inline'. Mirror the existing test in utils.test.ts by
calling makeAgent(generate) with structuredOutput.jsonPromptInjection set to
'system', forcing the retry path, and asserting the second generate call still
receives 'system' on structuredOutput.jsonPromptInjection.
In `@packages/core/src/stream/aisdk/v5/execute.test.ts`:
- Around line 115-122: The inline injection test in execute.test.ts only
verifies that the schema text is added to the last user message, but it does not
confirm the original user prompt is preserved. Update the test around the
capturedPrompt assertions in the v5 execute flow to also assert that the final
user turn still contains the original text from the latest user message (for
example, the prompt that was being appended to), ensuring the
append-to-latest-user-message behavior is validated rather than overwrite
behavior.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 677fd6da-caed-4ccf-b9e3-518583dc7a5f
📒 Files selected for processing (9)
.changeset/quick-dingos-eat.mdpackages/core/src/agent/durable/types.tspackages/core/src/agent/types.tspackages/core/src/agent/utils.test.tspackages/core/src/agent/utils.tspackages/core/src/features/index.tspackages/core/src/processors/processors/structured-output.tspackages/core/src/stream/aisdk/v5/execute.test.tspackages/core/src/stream/aisdk/v5/execute.ts
Co-Authored-By: Mastra Code (openai/gpt-5.5) <noreply@mastra.ai>
TylerBarnes
added a commit
that referenced
this pull request
Jun 30, 2026
## Summary
Adds the observational memory extractor pipeline.
Extractors let OM capture structured values alongside observations,
persist selected values into thread metadata, and carry extraction
failures through the OM lifecycle. Extractor instructions and schemas
can be static or resolved from runtime context, so extraction can adapt
to the active agent and thread.
```ts
observationalMemory: {
enabled: true,
observation: {
extract: [
new Extractor({
slug: 'priority',
name: 'Priority',
schema: z.enum(['low', 'medium', 'high']),
instructions: 'Extract the user priority when it changes.',
}),
],
},
}
```
## Test plan
- `pnpm --filter ./packages/memory test:unit -- --run
src/processors/observational-memory/__tests__/extractor.test.ts
src/processors/observational-memory/__tests__/observational-memory-api.test.ts
--reporter=dot --bail 1`
- `pnpm --filter ./packages/memory check`
- Stack rebase verified linear against #18652
## Stack
- Previous: #18652
- Next: #18654
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
ELI5: This PR makes “observational memory” not only remember what
happened, but also automatically pull out specific structured facts
(like a user’s priority), save them on the thread, and reuse them later.
It also adds an `Extractor` API so you can define how those facts are
detected, validated, and what happens when extraction fails.
### Summary
- Added a public `Extractor` API for Observational Memory with:
- **inline (XML/tag-based)** and **structured (Zod/schema-based)**
extraction modes
- slug validation, runtime-resolved/dynamic `instructions` and optional
`schema`
- per-extractor `onExtracted` hooks to normalize/transform values and
record failures safely.
- Extended Observational Memory configuration with extraction pipelines:
- `observationalMemory.observation.extract` and
`observationalMemory.reflection.extract`
- built-in extractors (current task, suggested response, thread title)
composed with user extractors.
- Implemented extraction end-to-end across observer/reflector:
- prompt generation includes extractor output sections and optional
“prior extracted values”
- parsing supports extracting values from model output while stripping
extractor sections
- structured extraction retries/fallback behavior is handled, and
extracted values are validated/merged.
- Persisted extracted values into thread observational memory metadata
under `threadOMMetadata.extracted`, including carry-forward of prior
extracted values into subsequent observer/reflector runs.
- Propagated extraction failures through the OM lifecycle (without
blocking successful extractions):
- added `extractionFailures` alongside `extractedValues` in OM end
markers and marker/telemetry payloads.
- Updated exports/types and documentation:
- new/extended types for `extractedValues` and `extractionFailures`
across OM configs, runners, and storage
- added Observational Memory docs sections for “Extractors” plus an
“Extractor API” reference.
- Added tests covering extractor logic and Observational Memory
persistence across:
- extractor unit behavior (inline/structured parsing, retries, hooks,
prompt composition)
- observational-memory API behavior (observe/reflect/buffer) and thread
metadata persistence
- marker schema changes and additional observational-memory lifecycle
assertions.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Mastra Code (crof/glm-5.2) <noreply@mastra.ai>
wardpeet
pushed a commit
that referenced
this pull request
Jul 1, 2026
## Summary
Adds a new `structuredOutput.jsonPromptInjection: 'inline'` mode.
Before this PR, JSON prompt injection put the structured-output schema
instructions in the system prompt (`true` / `'system'`). That works, but
it changes the system prompt for each schema and can defeat provider
prompt caching.
This PR adds an inline mode that appends the JSON instructions to the
latest user message instead, keeping the leading system prompt stable
and more cache-friendly.
```ts
await agent.generate('Extract the task details', {
structuredOutput: {
schema: taskSchema,
jsonPromptInjection: 'inline',
},
});
```
Behavior by mode:
```ts
jsonPromptInjection: true // inject into system prompt
jsonPromptInjection: 'system' // inject into system prompt
jsonPromptInjection: 'inline' // inject into latest user message
```
If there is no user message, inline mode adds a user message containing
the JSON instructions. Native `responseFormat` is still skipped whenever
prompt injection is enabled.
The structured-output fallback helpers preserve explicit `'inline'` /
`'system'` settings when retrying, instead of coercing every retry to
`true`.
## Test plan
- Stack rebase verified linear against #18651
- Covered by downstream memory/core checks in the stack
## Stack
- Previous: #18651
- Next: #18653
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## ELI5
This change lets the app add JSON “instructions” in a new “inline” way,
by attaching them to the user’s message instead of always putting them
in the system message. That helps keep the system prompt stable, which
can work better with providers that cache prompts.
## Summary
- Added support for `structuredOutput.jsonPromptInjection: 'inline'`
alongside `true` and `'system'`.
- Updated structured output handling so:
- `true` / `'system'` inject JSON instructions into the system prompt
- `'inline'` appends them to the latest user message
- if there is no user message, inline mode creates one
- Kept native `responseFormat` disabled whenever prompt injection is
enabled.
- Preserved explicit `'inline'` and `'system'` settings during JSON
fallback retries.
- Added the new runtime feature flag `json-prompt-injection:inline`.
- Expanded tests to cover inline injection, system injection,
missing-user-message behavior, and feature flag exposure.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Mastra Code (crof/glm-5.2) <noreply@mastra.ai>
wardpeet
pushed a commit
that referenced
this pull request
Jul 1, 2026
## Summary
Adds the observational memory extractor pipeline.
Extractors let OM capture structured values alongside observations,
persist selected values into thread metadata, and carry extraction
failures through the OM lifecycle. Extractor instructions and schemas
can be static or resolved from runtime context, so extraction can adapt
to the active agent and thread.
```ts
observationalMemory: {
enabled: true,
observation: {
extract: [
new Extractor({
slug: 'priority',
name: 'Priority',
schema: z.enum(['low', 'medium', 'high']),
instructions: 'Extract the user priority when it changes.',
}),
],
},
}
```
## Test plan
- `pnpm --filter ./packages/memory test:unit -- --run
src/processors/observational-memory/__tests__/extractor.test.ts
src/processors/observational-memory/__tests__/observational-memory-api.test.ts
--reporter=dot --bail 1`
- `pnpm --filter ./packages/memory check`
- Stack rebase verified linear against #18652
## Stack
- Previous: #18652
- Next: #18654
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
ELI5: This PR makes “observational memory” not only remember what
happened, but also automatically pull out specific structured facts
(like a user’s priority), save them on the thread, and reuse them later.
It also adds an `Extractor` API so you can define how those facts are
detected, validated, and what happens when extraction fails.
### Summary
- Added a public `Extractor` API for Observational Memory with:
- **inline (XML/tag-based)** and **structured (Zod/schema-based)**
extraction modes
- slug validation, runtime-resolved/dynamic `instructions` and optional
`schema`
- per-extractor `onExtracted` hooks to normalize/transform values and
record failures safely.
- Extended Observational Memory configuration with extraction pipelines:
- `observationalMemory.observation.extract` and
`observationalMemory.reflection.extract`
- built-in extractors (current task, suggested response, thread title)
composed with user extractors.
- Implemented extraction end-to-end across observer/reflector:
- prompt generation includes extractor output sections and optional
“prior extracted values”
- parsing supports extracting values from model output while stripping
extractor sections
- structured extraction retries/fallback behavior is handled, and
extracted values are validated/merged.
- Persisted extracted values into thread observational memory metadata
under `threadOMMetadata.extracted`, including carry-forward of prior
extracted values into subsequent observer/reflector runs.
- Propagated extraction failures through the OM lifecycle (without
blocking successful extractions):
- added `extractionFailures` alongside `extractedValues` in OM end
markers and marker/telemetry payloads.
- Updated exports/types and documentation:
- new/extended types for `extractedValues` and `extractionFailures`
across OM configs, runners, and storage
- added Observational Memory docs sections for “Extractors” plus an
“Extractor API” reference.
- Added tests covering extractor logic and Observational Memory
persistence across:
- extractor unit behavior (inline/structured parsing, retries, hooks,
prompt composition)
- observational-memory API behavior (observe/reflect/buffer) and thread
metadata persistence
- marker schema changes and additional observational-memory lifecycle
assertions.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Mastra Code (crof/glm-5.2) <noreply@mastra.ai>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a new
structuredOutput.jsonPromptInjection: 'inline'mode.Before this PR, JSON prompt injection put the structured-output schema instructions in the system prompt (
true/'system'). That works, but it changes the system prompt for each schema and can defeat provider prompt caching.This PR adds an inline mode that appends the JSON instructions to the latest user message instead, keeping the leading system prompt stable and more cache-friendly.
Behavior by mode:
If there is no user message, inline mode adds a user message containing the JSON instructions. Native
responseFormatis still skipped whenever prompt injection is enabled.The structured-output fallback helpers preserve explicit
'inline'/'system'settings when retrying, instead of coercing every retry totrue.Test plan
Stack
ELI5
This change lets the app add JSON “instructions” in a new “inline” way, by attaching them to the user’s message instead of always putting them in the system message. That helps keep the system prompt stable, which can work better with providers that cache prompts.
Summary
structuredOutput.jsonPromptInjection: 'inline'alongsidetrueand'system'.true/'system'inject JSON instructions into the system prompt'inline'appends them to the latest user messageresponseFormatdisabled whenever prompt injection is enabled.'inline'and'system'settings during JSON fallback retries.json-prompt-injection:inline.