Skip to content

feat(memory): add OM-managed working memory#18654

Merged
TylerBarnes merged 4 commits into
mainfrom
feat/working-memory-extractor
Jun 30, 2026
Merged

feat(memory): add OM-managed working memory#18654
TylerBarnes merged 4 commits into
mainfrom
feat/working-memory-extractor

Conversation

@TylerBarnes

@TylerBarnes TylerBarnes commented Jun 29, 2026

Copy link
Copy Markdown
Member

Summary

Adds a built-in WorkingMemoryExtractor and a simpler OM-managed working memory setup.

With manageWorkingMemory, OM can extract working-memory updates from observations and apply them after the turn. The sugar defaults working memory to prompt-cache-friendly state signals and disables direct agent tool management unless the user opts back in.

new Memory({
  options: {
    observationalMemory: {
      enabled: true,
      observation: {
        manageWorkingMemory: true,
        // extractors: [new WorkingMemoryExtractor()] <- added internally by manageWorkingMemory: true
      },
    },
    workingMemory: {
      enabled: true,
      // these are configured internally by observation.manageWorkingMemory: true
      // agentManaged: false, <- disables the agent managing working memory via tool calls and removes system instructions saying to do so
      // useStateSignals: true,
    },
  },
});

Users who still want both paths can opt in explicitly:

workingMemory: {
  enabled: true,
  agentManaged: true
}
Screenshot 2026-06-29 at 3 54 07 PM Screenshot 2026-06-29 at 3 25 57 PM

Test plan

  • pnpm --filter ./packages/memory test:unit -- --run src/processors/observational-memory/__tests__/extractor.test.ts src/processors/observational-memory/__tests__/observational-memory-api.test.ts --reporter=dot --bail 1
  • pnpm --filter ./packages/memory check
  • Stack rebase verified linear against feat(memory): add observational memory extractors #18653

Stack

ELI5

This PR helps Observational Memory automatically “catch” what changed in your app’s working memory as it watches the conversation, then apply those updates for the agent after the turn. So the agent doesn’t have to remember to call the working-memory tool every time—and the default setup is tuned to stay friendly to prompt caching.

Summary

  • Added a built-in WorkingMemoryExtractor and re-exported it from the Observational Memory module.
  • Introduced observationalMemory.observation.manageWorkingMemory to let Observational Memory extract working-memory updates and apply them after the turn.
  • Added workingMemory.agentManaged to control whether the main agent still receives working-memory tool/instructions (agentManaged defaults to true when not overridden).
  • When OM manages working memory (manageWorkingMemory: true with workingMemory.enabled: true), defaults change to:
    • workingMemory.agentManaged: false (OM applies updates instead of the agent)
    • workingMemory.useStateSignals: true (prompt-cache-friendly)
    • setting workingMemory.agentManaged: true preserves the “agent also manages it” path.
  • Updated memory behavior for tool exposure, system-message/tool instruction wording, and merged thread config defaults to match the new working-memory management rules.
  • Extended unit/integration tests to verify extraction and persistence for both markdown and JSON/schema-backed working memory modes, plus listTools()/getSystemMessage()/merged-config behavior under the new options.
  • Updated documentation and a changeset to document the new configuration flow and defaults (manageWorkingMemory, agentManaged, and useStateSignals).

@changeset-bot

changeset-bot Bot commented Jun 29, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 3a3e149

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 25 packages
Name Type
@mastra/memory Minor
@mastra/core Minor
mastracode Patch
@mastra/agent-builder Patch
@mastra/editor Patch
@internal/playground Patch
@mastra/express Patch
@mastra/fastify Patch
@mastra/hono Patch
@mastra/koa Patch
@mastra/nestjs Patch
@mastra/opencode Patch
@mastra/longmemeval Patch
@mastra/mcp-docs-server Patch
@mastra/client-js Patch
@mastra/server Minor
mastra Patch
@mastra/deployer-cloud Minor
@mastra/react Patch
@mastra/playground-ui Patch
@mastra/deployer Minor
create-mastra Patch
@mastra/next Patch
@mastra/tanstack-start Patch
@mastra/temporal Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel

vercel Bot commented Jun 29, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
mastra-docs-1.x Ready Ready Preview, Comment Jun 30, 2026 8:42pm
mastra-playground-ui Ready Ready Preview, Comment Jun 30, 2026 8:42pm
1 Skipped Deployment
Project Deployment Actions Updated (UTC)
mastra-studio-preview Ignored Ignored Preview Jun 30, 2026 8:42pm

Request Review

@coderabbitai

coderabbitai Bot commented Jun 29, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: fe91114a-848a-4d43-ada0-c8f87855ab4f

📥 Commits

Reviewing files that changed from the base of the PR and between 8564d66 and 3a3e149.

📒 Files selected for processing (1)
  • docs/src/content/en/docs/memory/observational-memory.mdx
✅ Files skipped from review due to trivial changes (1)
  • docs/src/content/en/docs/memory/observational-memory.mdx

Walkthrough

This PR adds an observationalMemory.observation.manageWorkingMemory option that enables Observational Memory to update working memory automatically through a new WorkingMemoryExtractor, with config defaults, tool/system-message behavior changes, tests, and documentation.

Changes

Observer-managed working memory

Layer / File(s) Summary
Config schema additions for agentManaged and manageWorkingMemory
packages/core/src/memory/types.ts, packages/memory/src/processors/observational-memory/types.ts
Adds agentManaged to working memory config and manageWorkingMemory to observational-memory observation config and serialized forms.
WorkingMemoryExtractor implementation and exports
packages/memory/src/processors/observational-memory/working-memory-extractor.ts, packages/memory/src/processors/observational-memory/index.ts, packages/memory/src/index.ts
Implements WorkingMemoryExtractor with config-derived details, instruction building, schema handling, and update logic, plus module re-exports.
Memory merged-config, tool list, and system message wiring
packages/memory/src/index.ts, packages/core/src/memory/mock.ts
Applies managed working-memory defaults in merged config, ensures the extractor is present, tightens updateWorkingMemory tool eligibility, adjusts read-only system message handling, and mirrors the tool logic in MockMemory.
Unit and integration tests for managed working memory
packages/memory/src/index.test.ts, packages/memory/src/processors/observational-memory/__tests__/extractor.test.ts, packages/memory/integration-tests/src/om-extracted-metadata.test.ts
Adds unit tests for tool listing and system message behavior, extractor hook tests, and end-to-end integration tests for markdown and schema-backed working memory updates.
Documentation and changeset for manageWorkingMemory
.changeset/honest-llamas-dress.md, docs/src/content/en/docs/memory/observational-memory.mdx, docs/src/content/en/docs/memory/working-memory.mdx, docs/src/content/en/reference/memory/observational-memory.mdx
Documents the option, its defaults, and usage examples across the changeset and memory docs.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~30 minutes

Suggested reviewers

  • abhiaiyer91
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title is concise and accurately summarizes the main change: OM-managed working memory for memory.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/working-memory-extractor

Comment @coderabbitai help to get the list of available commands.

@dane-ai-mastra

dane-ai-mastra Bot commented Jun 29, 2026

Copy link
Copy Markdown
Contributor

PR triage

Linked issue check skipped for core contributor @TylerBarnes.


PR complexity score

Factor Value Score impact
Files changed 13 +26
Lines changed 620 +36
Author merged PRs 570 -20
Test files changed Yes -10
Final score 32

Applied label: complexity: medium


Changed test gate

Changed tests passed against the base branch; they should fail before the PR code is applied.

Label: tests: failing ❌

@dane-ai-mastra dane-ai-mastra Bot added the tests: green ✅ Changed tests failed against base as expected label Jun 29, 2026
@TylerBarnes TylerBarnes force-pushed the feat/working-memory-extractor branch from 5d3d2b8 to 298e17e Compare June 29, 2026 23:57
Comment thread .changeset/honest-llamas-dress.md Outdated
Comment thread docs/src/content/en/docs/memory/observational-memory.mdx
Comment thread packages/memory/src/processors/observational-memory/working-memory-extractor.ts Outdated
Comment thread packages/memory/src/processors/observational-memory/working-memory-extractor.ts Outdated
Comment thread packages/memory/src/processors/observational-memory/working-memory-extractor.ts Outdated
Comment thread packages/memory/src/processors/observational-memory/working-memory-extractor.ts Outdated
@TylerBarnes TylerBarnes force-pushed the feat/working-memory-extractor branch from 298d8e0 to 230d9a5 Compare June 30, 2026 15:53
@dane-ai-mastra dane-ai-mastra Bot added tests: failing ❌ Changed tests passed against base and removed tests: green ✅ Changed tests failed against base as expected labels Jun 30, 2026
@dane-ai-mastra dane-ai-mastra Bot added the tests: green ✅ Changed tests failed against base as expected label Jun 30, 2026
@TylerBarnes TylerBarnes force-pushed the feat/working-memory-extractor branch from c504e9b to 78777fc Compare June 30, 2026 18:19
@dane-ai-mastra dane-ai-mastra Bot added tests: failing ❌ Changed tests passed against base and removed tests: green ✅ Changed tests failed against base as expected labels Jun 30, 2026
@dane-ai-mastra dane-ai-mastra Bot added tests: green ✅ Changed tests failed against base as expected and removed tests: failing ❌ Changed tests passed against base labels Jun 30, 2026
@TylerBarnes TylerBarnes force-pushed the feat/working-memory-extractor branch from 78777fc to 616b2bf Compare June 30, 2026 18:50
@dane-ai-mastra dane-ai-mastra Bot added tests: failing ❌ Changed tests passed against base and removed tests: green ✅ Changed tests failed against base as expected labels Jun 30, 2026
@dane-ai-mastra dane-ai-mastra Bot added tests: green ✅ Changed tests failed against base as expected and removed tests: failing ❌ Changed tests passed against base labels Jun 30, 2026
TylerBarnes added a commit that referenced this pull request Jun 30, 2026
## Summary

Adds the observational memory extractor pipeline.

Extractors let OM capture structured values alongside observations,
persist selected values into thread metadata, and carry extraction
failures through the OM lifecycle. Extractor instructions and schemas
can be static or resolved from runtime context, so extraction can adapt
to the active agent and thread.

```ts
observationalMemory: {
  enabled: true,
  observation: {
    extract: [
      new Extractor({
        slug: 'priority',
        name: 'Priority',
        schema: z.enum(['low', 'medium', 'high']),
        instructions: 'Extract the user priority when it changes.',
      }),
    ],
  },
}
```

## Test plan

- `pnpm --filter ./packages/memory test:unit -- --run
src/processors/observational-memory/__tests__/extractor.test.ts
src/processors/observational-memory/__tests__/observational-memory-api.test.ts
--reporter=dot --bail 1`
- `pnpm --filter ./packages/memory check`
- Stack rebase verified linear against #18652

## Stack

- Previous: #18652
- Next: #18654


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
ELI5: This PR makes “observational memory” not only remember what
happened, but also automatically pull out specific structured facts
(like a user’s priority), save them on the thread, and reuse them later.
It also adds an `Extractor` API so you can define how those facts are
detected, validated, and what happens when extraction fails.

### Summary
- Added a public `Extractor` API for Observational Memory with:
- **inline (XML/tag-based)** and **structured (Zod/schema-based)**
extraction modes
- slug validation, runtime-resolved/dynamic `instructions` and optional
`schema`
- per-extractor `onExtracted` hooks to normalize/transform values and
record failures safely.
- Extended Observational Memory configuration with extraction pipelines:
- `observationalMemory.observation.extract` and
`observationalMemory.reflection.extract`
- built-in extractors (current task, suggested response, thread title)
composed with user extractors.
- Implemented extraction end-to-end across observer/reflector:
- prompt generation includes extractor output sections and optional
“prior extracted values”
- parsing supports extracting values from model output while stripping
extractor sections
- structured extraction retries/fallback behavior is handled, and
extracted values are validated/merged.
- Persisted extracted values into thread observational memory metadata
under `threadOMMetadata.extracted`, including carry-forward of prior
extracted values into subsequent observer/reflector runs.
- Propagated extraction failures through the OM lifecycle (without
blocking successful extractions):
- added `extractionFailures` alongside `extractedValues` in OM end
markers and marker/telemetry payloads.
- Updated exports/types and documentation:
- new/extended types for `extractedValues` and `extractionFailures`
across OM configs, runners, and storage
- added Observational Memory docs sections for “Extractors” plus an
“Extractor API” reference.
- Added tests covering extractor logic and Observational Memory
persistence across:
- extractor unit behavior (inline/structured parsing, retries, hooks,
prompt composition)
- observational-memory API behavior (observe/reflect/buffer) and thread
metadata persistence
- marker schema changes and additional observational-memory lifecycle
assertions.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Mastra Code (crof/glm-5.2) <noreply@mastra.ai>
Base automatically changed from feat/om-extractors to main June 30, 2026 19:53
@dane-ai-mastra dane-ai-mastra Bot added complexity: critical Critical-complexity PR and removed complexity: medium Medium-complexity PR labels Jun 30, 2026
@TylerBarnes TylerBarnes force-pushed the feat/working-memory-extractor branch from 616b2bf to 3d64e85 Compare June 30, 2026 19:56
@dane-ai-mastra dane-ai-mastra Bot removed the complexity: critical Critical-complexity PR label Jun 30, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Nitpick comments (1)
packages/core/src/memory/mock.ts (1)

238-242: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

MockMemory.listTools still ignores readOnly unlike the real Memory.listTools.

The real Memory.listTools (packages/memory/src/index.ts) gates tool creation on workingMemoryConfig?.enabled && workingMemoryConfig.agentManaged !== false && !mergedConfig.readOnly. This mock's negated condition only covers enabled/agentManaged, so MockMemory will still expose updateWorkingMemory when readOnly is true, diverging from production behavior in tests that rely on the mock.

♻️ Proposed fix to align with real Memory.listTools
-    if (!mergedConfig.workingMemory?.enabled || mergedConfig.workingMemory.agentManaged === false) {
+    if (
+      !mergedConfig.workingMemory?.enabled ||
+      mergedConfig.workingMemory.agentManaged === false ||
+      mergedConfig.readOnly
+    ) {
       return {};
     }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/core/src/memory/mock.ts` around lines 238 - 242,
MockMemory.listTools is still missing the readOnly gate, so it can expose
updateWorkingMemory when the real Memory.listTools would not. Update the
conditional in MockMemory.listTools to match Memory.listTools behavior by also
checking mergedConfig.readOnly before returning the tool map, keeping the mock
aligned with the production logic in the listTools method.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@packages/memory/integration-tests/src/om-extracted-metadata.test.ts`:
- Around line 183-268: The mocked model output in this working-memory
observation test is missing the existing profile field, so the merge assertion
will not preserve the previous value. Update the mock in the observe flow used
by WorkingMemoryExtractor/doGenerate to include the full merged JSON for
“working-memory”, including profile.name along with the new location and
preferences, so getWorkingMemory() can assert the merged state correctly.

In
`@packages/memory/src/processors/observational-memory/__tests__/extractor.test.ts`:
- Around line 192-225: The test name and expectation are inconsistent with the
intended behavior in WorkingMemoryExtractor/onExtracted: once OM metadata is no
longer returned, this case should verify that the extractor does not persist
working-memory metadata in the hook result. Update the assertion in this test to
expect result.values to be undefined or omitted, and keep the title aligned with
the behavior verified by applyExtractorHooks and resolveExtractors.

In
`@packages/memory/src/processors/observational-memory/working-memory-extractor.ts`:
- Around line 71-93: The `onExtracted` hook in `working-memory-extractor` is
persisting working memory and then returning `current`, which causes
`applyExtractorHooks` to keep a duplicate `working-memory` value in
`extractedValues`. Make this hook side-effect-only by keeping the
`updateWorkingMemory` call and changing the return so it yields `undefined`
after persistence, while preserving the existing early exits for
`isSchemaWorkingMemory` and empty `workingMemory`.

---

Nitpick comments:
In `@packages/core/src/memory/mock.ts`:
- Around line 238-242: MockMemory.listTools is still missing the readOnly gate,
so it can expose updateWorkingMemory when the real Memory.listTools would not.
Update the conditional in MockMemory.listTools to match Memory.listTools
behavior by also checking mergedConfig.readOnly before returning the tool map,
keeping the mock aligned with the production logic in the listTools method.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 2bca477b-095d-4b26-a60e-7860e834c25c

📥 Commits

Reviewing files that changed from the base of the PR and between 6f578ac and 3d64e85.

📒 Files selected for processing (13)
  • .changeset/honest-llamas-dress.md
  • docs/src/content/en/docs/memory/observational-memory.mdx
  • docs/src/content/en/docs/memory/working-memory.mdx
  • docs/src/content/en/reference/memory/observational-memory.mdx
  • packages/core/src/memory/mock.ts
  • packages/core/src/memory/types.ts
  • packages/memory/integration-tests/src/om-extracted-metadata.test.ts
  • packages/memory/src/index.test.ts
  • packages/memory/src/index.ts
  • packages/memory/src/processors/observational-memory/__tests__/extractor.test.ts
  • packages/memory/src/processors/observational-memory/index.ts
  • packages/memory/src/processors/observational-memory/types.ts
  • packages/memory/src/processors/observational-memory/working-memory-extractor.ts

Comment thread packages/memory/integration-tests/src/om-extracted-metadata.test.ts Outdated
TylerBarnes and others added 4 commits June 30, 2026 13:21
Co-Authored-By: Mastra Code (crof/glm-5.2) <noreply@mastra.ai>
Adds a WorkingMemoryExtractor that lets Observational Memory update working memory through the normal extractor pipeline. The extractor reads the active working memory template or schema, writes updates through the memory instance, and does not duplicate the value under OM extracted metadata.

Also adds injectTools: boolean (default true) to BaseWorkingMemory config. Set to false when OM extractors or another path owns working memory updates, preventing the working memory tool from being injected into the main agent.

Co-Authored-By: Mastra Code (crof/glm-5.2) <noreply@mastra.ai>
Move the unreleased working memory extraction controls to the final agentManaged/manageWorkingMemory API. This keeps the working memory extractor branch from introducing the discarded injectTools name and documents the managed working memory defaults.

Co-Authored-By: Mastra Code (openai/gpt-5.5) <noreply@mastra.ai>
Addresses #18654 (comment)

Co-Authored-By: Mastra Code (openai/gpt-5.5) <noreply@mastra.ai>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

complexity: medium Medium-complexity PR Documentation Improvements or additions to documentation tests: green ✅ Changed tests failed against base as expected

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants